Image signal processing apparatus, and image signal processing method

ABSTRACT

An image signal generated by a CCD image sensor is processed by the block-generating section  28  provided in an image-signal processing section  25.  A class tap and a prediction tap are thereby extracted. The class tap is output to an ADRC process section  29,  and the prediction tap is output to an adaptation process section  31.  The ADRC process section  29  performs an ADRC process on the input image signal, generating characteristic data. A classification process section  30  generates a class code corresponding to the characteristic data thus generated and supplies the same to an adaptation process section  31.  The adaptation process section  31  reads, from a coefficient memory  32,  the set of prediction coefficients which corresponds to the class code. The set of prediction coefficients and the prediction tap are applied, thereby generating all color signals, i.e., R, G and B signals, at the positions of the pixels which are to be processed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of application Ser. No. 09/673,532, filed under35 U.S.C. §371, as the national phase of International ApplicationPCT/JP00/00950, having a claim of priority to Japanese Applications11-041114, filed in Japan on Feb. 19, 1999, 11-082228, filed in Japan onMar. 25, 1999, and 11-151859, filed in Japan on May 31, 1999, theentirety of which are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to an image-signal processing apparatus,an image-signal processing method, a learning apparatus, a learningmethod, and a recording medium. More particularly, the invention relatesto an image-signal processing apparatus, image-signal processing method,learning apparatus, a learning method and recording medium in which thecolor components of a pixel represented by an image signal areinterpolated by means of classification-adaptation process so that thepixel may have a red (R) component, a green (G) component and a blue (B)component.

BACKGROUND ART

There are two types of imaging apparatuses that have a solid-state imagesensor such as a CCD (Charge Coupled Device) image sensor. The firsttype has one CCD image sensor (hereinafter,_referred to as “single-platecamera”). The second type has three CCD image sensors (hereinafter,referred to as “three-plate camera”).

In a three-plate camera, the three CCD image sensors for generating an Rsignal, a G signal and a B signal, for example, generate threeprimary-color signals, respectively. The three primary-color signals areprocessed, whereby a color image signal is generated. The color imagesignal is recorded in a recording medium.

In a single-plate camera, the CCD image sensor is arranged in front of acolor-coding filter that comprises a color-filter array composed ofcolor filters, each allocated to one pixel. The color-coding filtergenerates color-coded, color component signals, each for one pixel. Thecolor-filter array, which constitutes the color-coding filter, includesprimary-color filters such as a R (Red) array, a G (Green) array and a B(Blue) array and complementary-color filters such as a Ye (Yellow)array, a Cy (Cyanogen) array and an Mg (Magenta) array. In thesingle-plate camera, the CCD image sensor generates a signalrepresenting one color for one pixel, and another signal representinganother color is generated for the pixel by means of linearinterpolation, thereby providing an image that is similar to the imagethe three-plate camera provides. The single-plate camera is incorporatedin a video camera or the like, which should be small and light.

The CCD image sensor provided in a single-plate camera may be arrangedin front of a color-coding filter that comprises a color-filter array ofsuch color arrangement as is shown in FIG. 1A. Each pixel of CCD imagesensor, arranged at the back of a color filter, outputs only an imagesignal that corresponds to the primary color R, G or B, of that colorfilter. That is, a pixel positioned at the back of an R filter outputsan R-component image signal, but cannot output a G-component imagesignal or a B-component image signal. Likewise, a G-component pixeloutputs only a G-component image signal, but cannot output anR-component image signal or a B-component image signal. A B-componentpixel outputs only a B-component image signal, but cannot output anR-component image signal or a G-component image signal.

The color arrangement of the color-filter array, which is shown in FIG.1A, is called “Bayer arrangement.” In this case, G-color filters arearranged in the pattern of a chessboard, and R-color filters and B-colorfilters are arranged in alternate columns, each in a vacant square.

An R-component signal, a G-component signal and a B-component signalmust be processed for each pixel in the next-stage section. To processthese signals, interpolation has hitherto been performed on the outputsfrom a CCD image sensor having n.times.m pixels (n and m are positiveintegers). Thus, n.times.m R-pixel image signals and n.times.m G-pixelimage signals are generated as is illustrated in FIG. 1B. That is, imagesignals equivalent to those CCD outputs of a three-plate camera areoutput to the next-stage section.

To generate image signals in, for example, density four times as high,interpolation is performed, generating 2n.times.2m R-pixel image signalsfrom the n.times.m R-pixel image signals, 2n.times.2m G-pixel imagesignals from the n.times.m G-pixel image signals, and 2n.times.2mB-pixel image signals from the n.times.m B-pixel image signals.

In the above-described single-lens camera, however, a linear process iscarried out to interpolate color signals. The waveform of the image isinevitably deformed, rendering the image unclear as a whole. A processsuch as edge emphasis must therefore be performed to increase theapparent resolution of the image. Since the image resolution achieved bythe image signals output from the single-plate camera is lower than theimage resolution attained by the outputs of a three-plate camera, theresultant image is blurred as a whole due to the influence of the linearprocess.

The three primary-color components of each pixel, which have the sameresolution, may be generated from an output of the CCD image sensor ofthe single-plate camera, thus obtaining image signals. Image signals maythen be obtained in a higher density from the image signals thusgenerated, thereby to increase the pixel density. This method, however,cannot provide a sufficient precision.

It is proposed that the classification-adaptation process, i.e., aprocess other than linear interpolation, be performed on the CCD outputsof the single-lens camera, for each of the R, G and B primary-colorimage signals, thereby to generate image signals that are equivalent tothe CCD outputs of a three-plate camera. (See Japanese PatentApplication No. 8-508623.) However, when the classification-adaptationprocess is effected on the R, G and B primary-color image signalsindependently, the same process is performed on each R pixel and each Bpixel as on each G pixel (two existing in very four pixels), though onlyone R pixel exists in very four pixels and only one B pixel exists invery four pixels in the m.times.n pixels as shown in FIGS. 1A and 1B inthe case where a color-filter array of Bayer arrangement is used.Consequently, high-precision prediction cannot be accomplished as far asthe R-component signals and B-component image signals are concerned.

DISCLOSURE OF THE INVENTION

Accordingly, an object of the present invention is to provide animage-signal processing apparatus, an image-signal processing method, alearning apparatus, a learning method, and a recording medium, whichenables single-plate cameras to generate image signals comparable withthose generated as image outputs by three-plate cameras, by performing aclassification-adaptation process to predict color signals.

Another object of the invention is to provide an image-signal processingapparatus, an image-signal processing method, a learning apparatus, alearning method, and a recording medium, which can reliably generatehigh-definition image signals.

Still another object of this invention is to provide an image-signalprocessing apparatus, an image-signal processing method, a learningapparatus, a learning method, and a recording medium, which can predictcolor signals with high accuracy, thereby generating image signals ofhigh resolution.

According to the invention, there is provided, an image-signalprocessing apparatus for processing an input image signal, said inputimage signal having any one of various color components at a position ofeach pixel. The apparatus comprises: extraction means for extracting,for each pixel of interest of the input image signal, a plurality ofpixels located near each pixel of interest; class-determining means fordetermining a class from the pixels extracted by the extraction means;and pixel-generating means for generating a pixel at a position of thepixel of interest in accordance with the class determined by theclass-determining means, said pixel having a color component differentfrom at least the color component of the pixel of interest.

According to this invention, there is provided an image-signalprocessing method of processing an input image signal, said input imagesignal having any one of various color components at a position of apixel. The method comprises: an extraction step of extracting, for eachpixel of interest of the input image signal, a plurality of pixelslocated near each pixel of interest; a class-determining step ofdetermining a class from the pixels extracted in the extraction step;and a pixel-generating step of generating a pixel at a position of thepixel of interest in accordance with the class determined in theclass-determining step, said pixel having a color component differentfrom at least the color component of the pixel of interest.

According to the present invention, there is provided a recording mediumstoring a computer program designed to process an input image signal,said input image signal having any one of various color components at aposition of a pixel. The computer program comprising: an extraction stepof extracting a plurality of pixels located near each pixel of interestof the input image signal; a class-determining step of determining aclass from the pixels extracted in the extraction step; and apixel-generating step of generating a pixel at a position of the pixelof interest in accordance with the class determined in theclass-determining step, said pixel having a color component differentfrom at least the color component of the pixel of interest.

According to this invention, there is provided a learning apparatuswhich comprises: first pixel-extracting means for extracting a pluralityof pixels located near each pixel of interest of a student-image signalwhich has one color component at respective position of pixel;class-determining means for determining a class from the pixelsextracted by the first pixel-extracting means; second pixel-extractingmeans for extracting a plurality of pixels located near positionscorresponding to the position of the pixel of interest of thestudent-image signal, from a teacher-image signal which corresponds tothe student-image signal and which have a plurality of color componentsfor each pixel; and prediction-coefficient generating means forgenerating a set of prediction coefficients for each class, for use ingenerating an image signal corresponding to the teacher-image signalfrom an image signal corresponding to the student-image signal, inaccordance with values of the pixels extracted by the firstpixel-extracting means and second pixel-extracting means.

According to the invention, there is provide a learning method whichcomprises: a first pixel-extracting step of extracting a plurality ofpixels located near each pixel of interest of a student-image signalwhich has one color component at respective position of pixel; aclass-determining step of determining a class from the pixels extractedin the first pixel-extracting step; a second pixel-extracting step ofextracting a plurality of pixels located near positions corresponding tothe position of the pixel of interest of the student-image signal, froma teacher-image signal which corresponds to the student-image signal andwhich have a plurality of color components for each pixel; and aprediction-coefficient generating step of generating a set of predictioncoefficients for each class, for use in generating an image signalcorresponding to the teacher-image signal from an image signalcorresponding to the student-image signal, in accordance with values ofthe pixels extracted in the first pixel-extracting step and secondpixel-extracting step.

According to this invention, there is provided a recording mediumstoring a computer program designed to perform a learning process togenerate a set of prediction coefficients corresponding to a class. Theprogram comprises: a first pixel-extracting step of extracting aplurality of pixels located near each pixel of interest of astudent-image signal which has one color component at respectiveposition of pixel, a class-determining step of determining a class fromthe pixels extracted in the first pixel-extracting step; a secondpixel-extracting step of extracting a plurality of pixels located nearpositions corresponding to the position of the pixel of interest of thestudent-image signal, from a teacher-image signal which corresponds tothe student-image signal and which have a plurality of color componentsfor each pixel; and a prediction-coefficient generating step ofgenerating a set of prediction coefficients for each class, for use ingenerating an image signal corresponding to the teacher-image signalfrom an image signal corresponding to the student-image signal, inaccordance with values of the pixels extracted in the firstpixel-extracting step and second pixel-extracting step.

According to the invention, there is provided an image-signal processingapparatus for processing an input image signal, said input image signalhaving a prescribed number of sample values which constitute one imageand each of which represents any one of various colors at each pixel.The apparatus comprises: extraction means for extracting, for each pixelof interest of the input signal, a plurality of pixels located near eachpixel of interest; class-determining means for determining a class fromthe pixels extracted by the extraction means; and output image-signalgenerating means for generating an output image signal having moresample values than the prescribed number, for the various colors, byprocessing each pixel of the input image signal in accordance with theclass determined by the class-determining means.

According to the present invention, there is provided an image-signalprocessing method of processing an input image signal, said input imagesignal having a prescribed number of sample values which constitute oneimage and each of which represents any one of various colors. The methodcomprises: an extraction step of extracting, for each pixel of interestof the input image signal, a plurality of pixels located near each pixelof interest; a class-determining step of determining a class from thepixels extracted in the extraction step; and an output image-signalgenerating step of generating an output image signal having more samplevalues than the prescribed number, for the various colors, by processingeach pixel of the input image signal in accordance with the classdetermined in the class-determining step.

According to the invention, there is provided a recording medium storinga computer program designed to process an input image signal, said inputimage signal having a prescribed number of sample values whichconstitute one image and each of which represents any one of variouscolors. The computer program comprises: an extraction step ofextracting, for each pixel of interest of the input image signal, aplurality of pixels located near each pixel of interest; aclass-determining step of determining a class from the pixels extractedin the extraction step; and an output image-signal generating step ofgenerating an output image signal having more sample values than theprescribed number, for the various colors, by processing each pixel ofthe input image signal in accordance with the class determined in theclass-determining step.

According to the present invention, there is provided a learningapparatus which comprises: first pixel-extracting means for extracting aplurality of pixels located near each pixel of interest from astudent-image signal having a prescribed number of sample values whichconstitute one image and each of which represents any one of variouscolors at a position of a pixel, said pixel of interest being oneincluded in an image to be predicted, which has more sample values thanthe prescribed number; class-determining means for determining a classfrom the pixels extracted by the first pixel-extracting means; secondpixel-extracting means for extracting a plurality of pixels located nearpositions corresponding to the position of the pixel of interest, from ateacher-image signal which corresponds to the image signal to bepredicted and which have a plurality of color components for each pixel;and prediction-coefficient generating means for generating a set ofprediction coefficients for each class, for use in a prediction processfor generating an image signal corresponding to the teacher-imagesignal, from an image signal that corresponds to the student-imagesignal, in accordance with values of the pixels extracted by the firstpixel-extracting means and second pixel-extracting means.

According to the invention, there is provided a learning method whichcomprises: a first pixel-extracting step of extracting a plurality ofpixels located near each pixel of interest, from a student-image signalhaving a prescribed number of sample values which constitute one imageand each of which represents any one of various colors at a position ofa pixel, said pixel of interest being one included in an image to bepredicted, which has more sample values than the prescribed number; aclass-determining step of determining a class from the pixels extractedin the first pixel-extracting step; a second pixel-extracting step ofextracting a plurality of pixels located near positions corresponding tothe position of the pixel of interest, from a teacher-image signal whichcorresponds to the image signal to be predicted and which have aplurality of color components for each pixel; and aprediction-coefficient generating step of generating a set of predictioncoefficients for each class, for use in a prediction process forgenerating an image signal corresponding to the teacher-image signal,from an image signal that corresponds to the student-image signal, inaccordance with values of the pixels extracted in the firstpixel-extracting step and second pixel-extracting step.

According to the invention, there is provided a recording medium storinga computer program designed to perform a learning process in accordancewith a class. The computer program comprises: a first pixel-extractingstep of extracting a plurality of pixels located near each pixel ofinterest, from a student-image signal having a prescribed number ofsample values which constitute one image and each of which representsany one of various colors at a position of a pixel, said pixel ofinterest being one included in an image to be predicted, which has moresample values than the prescribed number; a class-determining step ofdetermining a class from the pixels extracted in the firstpixel-extracting step; a second pixel-extracting step of extracting aplurality of pixels located near positions corresponding to the positionof the pixel of interest, from a teacher-image signal which correspondsto the image signal to be predicted and which have a plurality of colorcomponents for each pixel; and a prediction-coefficient generating stepof generating a set of prediction coefficients for each class, for usein a prediction process for generating an image signal corresponding tothe teacher-image signal, from an image signal that corresponds to thestudent-image signal, in accordance with values of the pixels extractedin the first pixel-extracting step and second pixel-extracting step.

According to the present invention, there is provided an image-signalprocessing apparatus for processing an input image signal, said inputimage signal having any one of various color components at a position ofeach pixel. This apparatus comprises: extraction means for extracting,for each pixel of interest of the input image signal, a plurality ofpixels located near each pixel of interest, each said plurality ofpixels having a color component of the highest density of all colorcomponents; class-determining means for determining a class from thepixels extracted by the extraction means; and pixel-generating means forgenerating a pixel in accordance with the class determined by theclass-determining means, said pixel having a color component differentfrom at least the color component of the pixel of interest.

According to this invention, there is provided an image-signalprocessing method of processing an input image signal, said input imagesignal having any one of various color components at a position of eachpixel. The method comprises: an extraction step of extracting, for eachpixel of the interest of the input image signal, a plurality of pixelslocated near each pixel of interest, each said plurality of pixelshaving a color component of the highest density of all color components;a class-determining step of determining a class from the pixelsextracted in the extraction step; and a pixel-generating step ofgenerating a pixel in accordance with the class determined in theclass-determining step, said pixel having a color component differentfrom at least the color component of the pixel of interest.

According to the invention, there is provided a recording medium storinga computer program designed to process an input image signal, said inputimage signal having any one of various color components at a position ofeach pixel. The computer program comprises: an extraction step ofextracting, for each pixel of interest of the input image signal, aplurality of pixels located near each pixel of interest, each saidplurality of pixels having a color component of the highest density ofall color components; a class-determining step of determining a classfrom the pixels extracted in the extraction step; and a pixel-generatingstep of generating a pixel in accordance with the class determined inthe class-determining step, said pixel having a color componentdifferent from at least the color component of the pixel of interest.

According to this invention, there is provided a learning apparatuswhich comprises: first pixel-extracting means for extracting a pluralityof pixels located near each pixel of interest of a student-image signalwhich has any one of various color components at a position of eachpixel, each said plurality of pixels having a color component of thehighest density of all color components; class-determining means fordetermining a class from the pixels extracted by the firstpixel-extracting means; second pixel-extracting means for extracting aplurality of pixels located near positions corresponding to the positionof the pixel of interest of the student-image signal, from ateacher-image signal which corresponds to the student-image signal andwhich have a plurality of color components for each pixel; andprediction-coefficient generating means for generating a set ofprediction coefficients for each class, for use in generating an imagesignal corresponding to the teacher-image signal from an image signalcorresponding to the student-image signal, in accordance with values ofthe pixels extracted by the first pixel-extracting means and secondpixel-extracting means.

According to the present invention, there is provided a learning methodwhich comprises: a first pixel-extracting step of extracting a pluralityof pixels located near each pixel of interest of a student-image signalwhich has any one of various color components at a position of eachpixel, each said plurality of pixels having a color component of thehighest density of all color components; a class-determining step ofdetermining a class from the pixels extracted in the firstpixel-extracting step; a second pixel-extracting step of extracting aplurality of pixels located near positions corresponding to the positionof the pixel of interest of the student-image signal, from ateacher-image signal which corresponds to the student-image signal andwhich have a plurality of color components for each pixel; and aprediction-coefficient generating step of generating a set of predictioncoefficients for each class, for use in generating an image signalcorresponding to the teacher-image signal from an image signalcorresponding to the student-image signal, in accordance with values ofthe pixels extracted in the first pixel-extracting means and secondpixel-extracting step.

According to this invention, there is provided a recording mediumstoring a computer program designed to perform a learning process inaccordance with a class. The computer program comprises: a firstpixel-extracting step of extracting a plurality of pixels located neareach pixel of interest of a student-image signal which has any one ofvarious color components at a position of each pixel, each saidplurality of pixels having a color component of the highest density ofall color components; a class-determining step of determining a classfrom the pixels extracted in the first pixel-extracting step; a secondpixel-extracting step of extracting a plurality of pixels located nearpositions corresponding to the position of the pixel of interest of thestudent-image signal, from a teacher-image signal which corresponds tothe student-image signal and which have a plurality of color componentsfor each pixel; and a prediction-coefficient generating step ofgenerating a set of prediction coefficients for each class, for use ingenerating an image signal corresponding to the teacher-image signalfrom an image signal corresponding to the student-image signal, inaccordance with values of the pixels extracted in the firstpixel-extracting means and second pixel-extracting step.

According to the invention, there is provided an image-signal processingapparatus for processing an input image signal, said input image signalhaving any one of various color components at a position of each pixel.The apparatus comprises: extraction means for extracting, for each pixelof interest of the input image signal, a plurality of pixels for eachcolor component, from pixels located near each pixel of interest;class-determining means including a characteristic-data generatingsection for generating characteristic data about the pixels of eachcolor component, from the pixels of each color component which have beenextracted by the extraction means, and a class-determining section fordetermining a class from the characteristic data generated for eachcolor component; and pixel-generating means for generating a pixel inaccordance with the class determined by the class-determining means,said pixel having a color component different from at least the colorcomponent of the pixel of interest.

According to this invention, there is provided an image-signalprocessing method of processing an input image signal, said input imagesignal having any one of various color components at a position of eachpixel. The apparatus comprises: an extraction step of extracting, foreach pixel of interest of the input image signal, a plurality of pixelsfor each color component, from pixels located near each pixel ofinterest; a class-determining step of generating characteristic dataabout the pixels of each color component, from the pixels of each colorcomponent which have been extracted in the extraction step anddetermining a class from the characteristic data generated for eachcolor component; and a pixel-generating step of generating a pixel inaccordance with the class determined in the class-determining step, saidpixel having a color component different from at least the colorcomponent of the pixel of interest.

According to the invention, there is provided a recording medium storinga computer program designed to process an input image signal, said inputimage signal having any one of various color components at a position ofeach pixel. The computer program comprises: an extraction step ofextracting, for each pixel of interest of the input image signal, aplurality of pixels for each color component, from pixels located neareach pixel of interest; a class-determining step of generatingcharacteristic data about the pixels of each color component, from thepixels of each color component which have been extracted in theextraction step and determining a class from the characteristic datagenerated for each color component; and a pixel-generating step ofgenerating a pixel in accordance with the class determined in theclass-determining step, said pixel having a color component differentfrom at least the color component of the pixel of interest.

According to the present invention, there is provided a learningapparatus which comprises: first pixel-extracting means for extracting aplurality of pixels for each color component, from pixels located neareach pixel of interest of a student-image signal having one colorcomponent at respective position of pixel; class-determining means forgenerating characteristic data about the pixels of each color component,from the pixels of each color component which have been extracted by thefirst pixel-extracting means and for determining a class from thecharacteristic data generated for each color component; secondpixel-extracting means for extracting a plurality of pixels located nearpositions corresponding-to the position of the pixel of interest of thestudent-image signal, from a teacher-image signal which corresponds tothe student-image signal and which have a plurality of color componentsfor each pixel; and prediction-coefficient generating means forgenerating a set of prediction coefficients for each class, for use ingenerating an image signal corresponding to the teacher-image signalfrom an image signal corresponding to the student-image signal, inaccordance with values of the pixels extracted by the firstpixel-extracting means and second pixel-extracting means.

According to this invention, there is provided a learning method whichcomprises: a first pixel-extracting step of extracting a plurality ofpixels for each color component, from pixels located near each pixel ofinterest of a student-image signal having one color component atrespective position of pixel; a class-determining step of generatingcharacteristic data about the pixels of each color component, from thepixels of each color component which have been extracted in the firstpixel-extracting step and for determining a class from thecharacteristic data generated for each color component; a secondpixel-extracting step of extracting a plurality of pixels located nearpositions corresponding to the position of the pixel of interest of thestudent-image signal, from a teacher-image signal which corresponds tothe student-image signal and which have a plurality of color componentsfor each pixel; and a prediction-coefficient generating step ofgenerating a set of prediction coefficients for each class, for use ingenerating an image signal corresponding to the teacher-image signalfrom an image signal corresponding to the student-image signal, inaccordance with values of the pixels extracted in the firstpixel-extracting step and second pixel-extracting step.

According to the present invention, there is provided a recording mediumstoring a computer program designed to perform a learning process inaccordance with a class. The computer program comprises: a firstpixel-extracting step of extracting a plurality of pixels for each colorcomponent, from pixels located near each pixel of interest of astudent-image signal having one color component at respective positionof pixel; a class-determining step of generating characteristic dataabout the pixels of each color component, from the pixels of each colorcomponent which have been extracted in the first pixel-extracting stepand for determining a class from the characteristic data generated foreach color component; a second pixel-extracting step of extracting aplurality of pixels located near positions corresponding to the positionof the pixel of interest of the student-image signal, from ateacher-image signal which corresponds to the student-image signal andwhich have a plurality of color components for each pixel; and aprediction-coefficient generating step of generating a set of predictioncoefficients for each class, for use in generating an image signalcorresponding to the teacher-image signal from an image signalcorresponding to the student-image signal, in accordance with values ofthe pixels extracted in the first pixel-extracting step and secondpixel-extracting step.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A, FIG. 1B and FIG. 1C are diagrams schematically illustrating theconventional processing of image signals by means of the linearinterpolation;

FIG. 2 is a block diagram showing the structure of a digital stillcamera to which the present invention is applied;

FIG. 3 is a flowchart for explaining how the digital still cameraoperates;

FIG. 4 is a block diagram showing the structure of the image signalprocessing section incorporated in the digital still camera;

FIG. 5 is a flowchart for explaining how the image signal processingsection processes image signals;

FIG. 6 is a block diagram showing the section for effecting aclassification-adaptation process to predict color signals;

FIG. 7 is a block diagram depicting a section designed to determine aset of prediction coefficients;

FIG. 8 is a schematic representation of the structure of a predictiontap;

FIGS. 9A, 9B, 9C and 9D are diagrams illustrating how the image signalprocessing section performs the classification-adaptation process,thereby to process image signals;

FIGS. 10A and 10B are diagrams showing a set of prediction coefficients;

FIGS. 11A and 11B are diagrams illustrating another set of predictioncoefficients;

FIGS. 12A and 12B are diagrams showing another set of predictioncoefficients;

FIG. 13 is a block diagram showing a learning apparatus that acquiressets of prediction coefficients by learning;

FIG. 14 is a flowchart for explaining how the learning apparatusoperates;

FIGS. 15A, 15B and 15C are diagrams schematically showing how thelearning apparatus operates;

FIGS. 16A, 16B and 16C are diagrams depicting class taps;

FIGS. 17A, 17B and 17C are diagrams showing class taps;

FIGS. 18A, 18B and 18C are diagrams illustrating class taps;

FIGS. 19A, 19B and 19C are diagrams showing class taps;

FIGS. 20A to 20N are schematic representations of various color-filterarrays, each comprising color-coding filters, which may be used in theCCD image sensor incorporated in the digital still camera;

FIGS. 21A, 21B and 21C are diagrams illustrating another type of imagesignal processing the image signal processing section may perform;

FIGS. 22A and 22B are diagrams showing still another of image signalprocessing the image signal processing section may perform;

FIG. 23 is a diagram showing a class tap;

FIG. 24 is a diagram showing another type of a class tap;

FIG. 25 is a diagram depicting still another type of a class tap;

FIG. 26 is a diagram showing another type of a class tap;

FIG. 27 is a diagram illustrating a prediction tap;

FIG. 28 is a diagram showing another type of a prediction tap;

FIG. 29 is a diagram depicting another type of a prediction tap;

FIG. 30 is a diagram illustrating a different type of a prediction tap;

FIG. 31 is a diagram showing a pixel predicted;

FIG. 32 is a diagram schematically showing a class tap that correspondsto the pixel shown in FIG. 31;

FIG. 33 is a diagram illustrating a prediction tap that corresponds tothe pixel shown in FIG. 31;

FIG. 34 is a diagram showing a pixel predicted;

FIG. 35 is a diagram schematically showing a class tap that correspondsto the pixel shown in FIG. 34;

FIG. 36 is a diagram illustrating a prediction tap that corresponds tothe pixel shown in FIG. 34;

FIG. 37 is a diagram showing a pixel predicted;

FIGS. 38A and 38B are diagrams schematically showing class taps thatcorrespond to the pixel shown in FIG. 37;

FIGS. 39A and 39B are diagrams schematically showing class taps thatcorrespond to the pixel shown in FIG. 37;

FIGS. 40A, 40B and 40C illustrate a tap of another type;

FIGS. 41A, 41B and 41C show a tap of still another type; and

FIG. 42 is a block diagram of a computer system.

BEST MODE FOR CARRYING OUT THE INVENTION

The best mode for carrying out the present invention will be describedin detail, with reference to the accompanying drawings.

The present invention is applied to, for example, a digital still camera1 of the type shown in FIG. 2. The digital still camera 1 is asingle-plate camera designed to taking color pictures. The camera 1comprises a color-coding filter 4 and one CCD image sensor 5. Thecolor-coding filter 4 is arranged in front of the CCD image sensor 5 andcomposed of color filers, each provided for one pixel. A lens 2 focusesthe light reflected from a picked image. The light is applied to the CCDimage sensor 5 through an iris 3 and the color-coding filter 4. An imageof the object is thereby formed on the image-forming surface of the CCDimage sensor 5. The color-coding filter 4 and the CCD image sensor 5 arecomponents separated from each other in the digital still camera 1.Nonetheless, they may be combined into one unit.

The CCD image sensor 5 receives light for a time determined by theoperation of the shutter that is controlled by a timing signal suppliedfrom a timing generator 9. The sensor 5 generates a signal charge(analog value) for each pixel, which corresponds to the amount of lightthat has passed through the color-coding filter 4. An image signalrepresenting the image of the object, which the light incident to thesensor 4 has formed, is thereby generated. The image signal is supplied,as image output, to a signal-adjusting section 6.

The signal-adjusting section 6 comprises an AGC (Automatic Gain Control)circuit and a CDS (Correlated Double Sampling) circuit. The AGC circuitadjusts the gain to set the image signal at a constant level. The CDScircuit removes the 1/f noise made by the CCD image sensor 5.

The image signal output from the signal-adjusting section 6 is suppliedto an A/D converter section 7, which converts the image signal to adigital signal. The digital signal is supplied to an image-signalprocessing section 8. The A/D converter section 7 generates a digitalimage signal in accordance with a timing signal supplied from the timinggenerator 9. The digital image signal has, for example, 10 bits for eachsample.

In the digital still camera 1, the timing generator 9 supplies varioustiming signals to the CCD image sensor 5, signal-adjusting section 6,A/D converter section 7 and CPU 10. The CPU 10 drives a motor 11,controlling the iris 3. The CPU 10 drives a motor 12, moving the lens 2and the like, performing controls such as zooming and focusing. Further,the CPU 10 controls a flash lamp 13, whenever necessary, to applyflashing light to the object.

The processing section 8 performs defect-eliminating process, digitalclamping process, white-balance adjusting process, gamma-correctingprocess, prediction process using classification adaptation, and thelike, on the image signal supplied from the A/D converter section 7.

A memory 15 is connected to the image signal-processing section 8. Thememory 15 is, for example, a RAM (Random Access Memory) and provided forstoring signals that are necessary for the section 8 to process theimage signal. The image signal processed by the image-signal processingsection 8 is stored via an interface 14 into a memory 16. The imagesignal is supplied from the memory 16 via the interface 14 and recordedin a recording medium 17 that can be removably inserted in the digitalstill camera 1,

The motor 11 drives the iris 3 in accordance with the control datasupplied from the CPU 10. Thus driven, the iris 3 controls the amount ofincident light passing through the lens 2. The motor 12 moves the lens 2toward and away from the CCD image sensor 2 in accordance with thecontrol data supplied from the CPU 10, whereby the focusing condition iscontrolled. Automatic iris control, automatic focusing control and thelike can thereby accomplished. The flash lamp 13 emits flashing light tothe object in a predetermined amount, under the control of the CPU 10.

The interface 14 supplies the image signal output from the image-signalprocessing section 8, into the memory 16 if necessary. The interface 14also effects interface process on the image signal and then supplies thesignal to the recording medium 17 so that the signal may be recorded inthe medium 17. The recording medium 17 is one that can be removably setin the main body of the digital still camera 1. It may be a floppy disk,a disk-shaped recording medium for use in hard disk drives, a flashmemory such as a memory card, or the like.

A controller 18 is provided, which is controlled by the CPU 10 to supplycontrol data to the image-signal processing section 8 and the interface14, thereby controlling the section 8 and the interface 14. To the CPU10 there are input operation data generated as the user operates anoperation section 20 that has buttons including a shutter button and azoom button. In accordance with the operation data, the CPU 10 controlsthe above-mentioned other components of the camera 1. The camera 1 has apower supply section 19, which comprises a battery 19A and a DC/DCconverter 19B. The DC/DC converter 19B converts the power supplied fromthe battery 19A, to a direct current of a predetermined value, which issupplied to the various components of the camera 1. The battery 19A,which is a rechargeable one, is removably set in the main body of thedigital still camera 1.

How the digital still camera 1 shown in FIG. 2 operates will bedescribed, with reference to the flowchart of FIG. 3. The digital stillcamera 1 starts picking the image in Step S1 when the power switch isturned on. More precisely, the CPU 10 drives the motor 11 and the motor12, achieving the focusing and adjusting the iris 3. The light reflectedfrom the object is applied through the lens 2, forming an image of theobject on the CCD image sensor 5.

In Step S2, the signal-adjusting section 6 adjusts the gain of the imagesignal generated by the CCD image sensor 5 and representing the imageformed on the sensor 5, thereby setting the gain at the constant level.Further, the section 6 removes the noise from the image signal. The A/Dconverter section 7 converts the image signal to a digital signal.

In Step S3, the image-signal processing section 8 performs processes,including classification-adaptation process, on the digital image signalgenerated by the A/D converter section 7.

The user can review the picked image since the image represented by thesignal output from the CCD image sensor 5 is displayed in a view finder.The image of the object may be reviewed by the user through an opticalview finder.

If the user wants to record the image of the object he or she hasreviewed through the view finder, he or she operates the shutter buttonprovided on the operation section 20. In Step S4, the CPU 10 of thedigital still camera determines whether the shutter button has beenoperated. The digital camera 1 repeats Steps S2 and S3 until the CPU 10determines that the shutter button has been operated. When the CPU 10determines that the shutter button has been operated, the operation goesto Step S5.

In Step S5, the image signal, which the image-signal processing section8 has processed, is supplied via the interface 14 to the recordingmedium 17. The image signal is thereby recorded in the recording medium17.

The image-signal processing section 8 will be described, with referenceto FIG. 4.

The image-signal processing section 8 has a defect correcting section21, which receives the digital image signal from the A/D convertersection 7. The image signal may contain pixel signals generated fromthose pixels of the CCD image sensor 5 which do not react to theincident light or which always hold electric charges. The defectcorrecting section 21 detects such pixel signals, or defective pixelsignals, and processes the image signal to prevent the defective pixelsignals from adversely influencing the quality of the image signal.

In the A/D converter section 7, the value of input analog signal isshifted to the positive-value side in order not to cut the negativevalue, and the analog signal is converted to a digital signal. Aclamping section 22 clamps the image signal the defect correctingsection 21 has corrected, thereby eliminating the shifted component ofthe image signal.

The image signal clamped by the clamping section 22 is supplied to awhite-balancing section 23. The white-balancing section 23 corrects thegain of the image signal supplied from the clamping section 22, thusadjusting the white balance of the image signal. The image signal havingits white balance adjusted is supplied to a gamma-correcting section 24.The gamma-correcting section 24 corrects the level of the image signalwhose white balance has been corrected by the white-balance correctingsection 23, in accordance with a gamma curve. The image signal, thusgamma-corrected, is supplied to a prediction-process section 25.

The prediction-process section 25 carries out aclassification-adaptation process, converting the output of thegamma-correcting section 24 to an image signal comparable with, forexample, a CCD output of a three-plate camera. The image signal thusgenerated is supplied to a signal-correcting section 26. Theprediction-process section 25 comprises a block-generating section 28,an ADRC (Adaptive Dynamic Range Coding) section 29, a classificationprocess section 30, an adaptation process section 31, a coefficientmemory 32, and the like.

The block-generating section 28 supplies a class-tap image signal(described later) to the ADRC process section 29 and a prediction-tapimage signal to the adaptation process section 31. The ADRC processsection 29 performs ADRC process on the class-tap image signal input toit, thereby generating a re-quantized code. The re-quantized code issupplied, as characteristic data, to the classification process section30. The classification process section 30 classifies the pattern of theimage signal on the basis of the characteristic data supplied from theADRC process section 29. The section 30 generates a class number (classcode) that represents the result of classification. The coefficientmemory 32 supplies the adaptation process section 31 with the set ofcoefficients, which corresponds to the class number generated by theclassification process section 30. The adaptation process section 31uses the set of coefficients supplied from the coefficient memory 32,calculating a predicted pixel value from the prediction-tap image signalsupplied from the block-generating section 28.

The signal-correcting section 26 performs so-called picture-formingprocess, such as edge emphasis, on the image signal processed by theprediction-process section 25, thereby to improve the visualcharacteristics of the image.

A color-space converting section 27 is provided, which performs matrixconversion on the image signal (RGB signal) that has been subjected toedge emphasis in the prediction-process section 25. The image signal isthereby converted to an image signal of a prescribed format, such as aYUV format (composed of luminance Y and color differences U and V).Alternatively, the color-space converting section 27 may output the RGBsignal, without performing the matrix conversion on the RGB signal. Inthis embodiment of the invention, either a YUV signal or a RGB signalcan be output, as the user operates an operation section 20. The imagesignal output from the color-space converting section 27 is supplied tothe interface 14 described above.

How the image-signal processing section 8 processes an image signal inStep S3 shown in the flowchart of FIG. 3 will be described, withreference to the flowchart of FIG. 5.

In the image-signal processing section 8, the A/D converter section 7starts processing a digital image signal. First, in Step S11, the defectcorrecting section 21 corrects the image signal, eliminating the defectsin the image signal so that the image signal may not be adverselyinfluenced by the defects. In Step S12, the clamping section 22 clampsthe image signal that the defect correcting section 21 has corrected,thereby eliminating that component of the image signal which has beenshifted to the right.

In Step S13, the white-balancing section 23 corrects the white balanceof the image signal that has been clamped by the clamping section 22,thereby adjusting the gains of the color signals. Further, in Step S14,the gamma-correcting section 24 corrects the level of the image signalwhose white balance has been corrected by the white-balance correctingsection 23, in accordance with a gamma curve.

In Step S15, a classification-adaptation process is effected to predictcolor signals. This step consists of Steps S151 to S155. In Step S151,the block-generating section 28 processes the image signal thegamma-correcting section 24 has corrected, thus generating a block. Inother words, the section 28 extracts a class tap and a prediction tapfrom the image signal. The class tap contains pixels corresponding tovarious types of color signals.

In Step S152, the ADRC process section 29 performs an ADRC process.

In Step S153, the classification process section 30 classifies thepattern of the image signal on the basis of the characteristic datasupplied from the ADRC process section 29. The section 30 supplies theclass number assigned to the pattern classified, to the adaptationprocess section 31.

In Step S154, the adaptation process section 31 reads the set ofcoefficients corresponding to the class number supplied from theclassification process section 30, from the coefficient memory 32. Thesection 31 multiplies the coefficients of the set by the image signalsthat correspond to the coefficients. The coefficient-signal products areadded together, thereby predicting pixel values.

In Step 155, it is determined whether or not the processes have beenperformed on the entire region. If it is determined that the processeshave been performed on all regions, the operation goes to Step S 16. Ifnot, the operation goes to Step S151 and the next region will beprocessed.

In Step S16, a correction process (so-called “picture-forming process”)is effected on the image signal, which has been generated in Step S15and which is comparable with one output by the CCD of a three-platecamera, so that the visual characteristics of the resultant image may beimproved. In Step S17, color-space conversion is performed on the imagesignal obtained in Step S16. For example, a RGB signal is converted to aYUV signal.

The classification-adaptation process will be explained. FIG. 6 shows asection of ordinary type that effects a classification-adaptationprocess. In this section, the input image is supplied toregion-extracting sections 101 and 102. The region-extracting section101 extracts an image region (called “class tap”) from the input imagesignal and supplies the image region to an ADRC process section 103. TheADRC process section 103 performs ADRC process on the signal supplied toit, thus generating a quantized code.

The ADRC process is carried out to generate the quantized code. Instead,DCT (Discrete Cosine Transform), VQ (Vector Quantization), BTC (BlockTrancation Coding), non-linear quantization, or the like may beperformed to generate the quantized code.

ADRC is an adaptive re-quantization method that has been developed toachieve high-efficiency encoding in VTRs (Video Tape Recorders). Themethod is advantageous in that a local signal-level pattern can beexpressed in small amount of data. Thanks to this advantage, the methodcan be utilized to detect a pattern in the time-space, i.e., spaceactivity, of an image signal. The ADRC process section 103 re-quantizesthe class tap, i.e., the region extracted. More precisely, it dividesthe difference between the maximum value MAX and minimum value MIN inthe class tap, by the number of bits designated, in accordance with thefollowing equations:DR=MAX−MIN+1Q=[(L−MIN+0.5).times.2n/DR]  (1)

where DR is the dynamic range of the region and n is the number of bitsallocated to the region, L is the signal level of pixels present in theregion, and Q is the re-quantized code. For example, n may be 2, thatis, n=2. The bracket ([ . . . ]) means the process of omitting thedecimal fractions.

Thus, an image-signal class tap in which each pixel consists of, forexample, eight bits is converted to a 2-bit re-quantized code. There-quantized code, thus generated, represents the level-distributionpattern in the image-signal class tap, by using a small amount ofinformation. If the class tap is composed of, for example, seven pixels,the above-mentioned process is performed, thereby generating sevenre-quantized codes q1 to q7. The class code is of such a type as givenby the following equation (2): 1 class=i=1 q i (2 p) i (2)

where n is the number of pixels to be extracted as a class tap. Thevalue for p may be 2, that is, p=2.

The class code, class, is characteristic data that represents the spaceactivity, i.e., the level-distribution pattern in the time-space of theimage signal. The class code, class, is supplied to a predictioncoefficient memory 104. The prediction coefficient memory 104 storessets of prediction coefficients, each set assigned to one class as willbe described later. The memory 104 outputs the set of predictioncoefficients that corresponds to the class identified by there-quantized code supplied to the memory 104. In the meantime, theregion-extracting sections 102 extracts a region of the image (calledredicted tap and supplies the image signal of the predicted tap to aprediction section 105. The prediction section 105 performs theoperation of the following equation (3) on the set of coefficientssupplied from the prediction coefficient memory 104, thereby generatingan output image signal y.y=w.sub.1.times.x.sub.1+w.sub.2.times.x.sub.2+ . . .+w.sub.n.times.x.sub.n   (3)

where x.sub.1, and xn are the values of the pixels constituting thepredicted tap, and w1, and wn are the prediction coefficients.

The process of determining which set of prediction coefficients shouldbe applied will be explained, with reference to FIG. 7. A HD(High-Definition) image signal having the same image format as theoutput image signal is supplied to a HD-SD converter section 201 and apixel-extracting section 208. The HD-SD converter section 201 carriesout extraction or the like, whereby the HD image signal is converted toan image signal (hereinafter referred to as “SD (Standard-Definition)image signal”) that has a resolution (i.e., number of pixels) similar tothat of the input image signal. The SD image signal is supplied to aregion-extracting sections 202 and 203. Like the region-extractingsection 101 described above, the region-extracting section 202 extractsa class tap from the SD image signal and generates an image signalrepresenting the class tap. This signal is supplied to an ADRC processsection 204.

The ADRC process section 204 performs the same ADRC process as does theADRC process section 103 shown in FIG. 6. That is, the section 204performs ADRC process, thus generating a re-quantized code from thesignal supplied to it. The re-quantized code is supplied to a class-codegenerating section 205. The class-code generating section 205 isgenerates a class code that represents the class of the re-quantizedcode supplied to the section 205. The class code is supplied to anormal-equation adding section 206. Meanwhile, the region-extractingsection 203 extracts a predicted tap from the SD image signal in thesame way as does the region-extracting section 102 shown in FIG. 6. Theimage signal representing this predicted tap is supplied to thenormal-equation adding section 206.

The normal-equation adding section 206 adds the image signals suppliedfrom the region-extracting sections 203 to the image signals suppliedfrom a pixel-extracting section 208, for each class code supplied fromthe class-code generating section 205. The signals, thus added for eachclass code, are supplied to a prediction-coefficient determining section207. On the basis of the signals for each class code, theprediction-coefficient determining section 207 selects predictioncoefficients that constitute a set to be applied to the class code.

The operation that is performed to determine a set of predictioncoefficients will be explained. Prediction coefficient w1 is calculatedas will be described, by supplying various image signals, or HD imagesignals, to the section illustrated in FIG. 7. Assuming that the numberof the image signals supplied is m, we obtain the following equation (4)from the equation (3):y.sub.k=w.sub.1x.sub.k1+w.sub.2.times.x.sub.k2+ . . .+w.sub.n.times.x.sub.kn   (4)

If m>n, coefficients w.sub.1, . . . , and wn cannot be determineddirectly. Thus, the element ek of an error vector e is defined as shownin the following equation (5), thereby to determine a set of predictioncoefficients so that the square of the error vector e defined by theequation (6) presented below may become minimal. That is, so-called“least squares method” is applied, thereby directly determining a set ofprediction coefficients. 2 e k=y k−{w 1 .times. x k1+w 2.times. k 2++ wn .times. k n} (k=1, 2, m)(5) 2=k=0 m e k 2 (6)

A method that may actually used to obtain a set of predictioncoefficients, which minimizes the value e.sup.2 in the equation (6), isto find partial differential values of e.sup.2 for the predictioncoefficients w.sub.i (i=1, 2, . . . ), as indicated by the followingequation (7). In the method it suffices to determine each predictioncoefficient to impart a partial differential value of 0 to theprediction coefficient. 3 2 w i=k=0 m 2 [e k w i] e k=k=0 m 2 X ki e k(7)

The sequence of operations for determining the prediction coefficientsby using the equation (7) will be described. Let X.sub.ji and Y.sub.i bedefined by the equations (8) and (9). The equation (7) can then berewritten to the following equation (10): 4 X ij=p=0 m X pi X pj (8) Yi=k=0 m y ki y k (9) [X 11 X 12 X 1 n X 21 X 22 X 2 n X n1 X n2 X nn] [W1 W 2 W n]=[Y 1 Y 2 Y n](10)

The equation (10) is generally called “normal equation.” Thenormal-equation adding section 205 carries out operations of theequations (8) and (9) on the signal supplied to it, thereby calculatingX.sub.ji and Yi(i=1, 2, . . . , n). The prediction-coefficientdetermining section 207 solves the normal equation (10) by using anordinary matrix analysis, thereby calculating prediction coefficientsw.sub.i(i=1, 2,.

In this instance, the set of prediction coefficients and the predictiontap, which correspond to the characteristic data of a pixel of interest,are applied, performing operations on a linear combination model. Anadaptation process is thereby performed. The set of predictioncoefficients, which is used to perform the adaptation process, can beacquired by learning. Alternatively, the adaptation process can beaccomplished by using the pixel values corresponding to the class or byeffecting operations on the linear combination model.

The classification-adaptation process described above achieves variousconversions of image signals, which generate noise-free image signalsfrom input image signals, image signals having their scanning-lineschemes converted, and the like. These signals generated are used asoutput image signals.

In the digital still camera 1, a plurality of pixels near each pixel ofinterest are extracted from an input image signal that has any one ofthe color components which the CCD image sensor of the camera 1(single-plate camera) has generated.

The class is determined on the basis of the pixels thus extracted. Thenormal-equation adding section 205 performs theclassification-adaptation process, thus generating, at the position ofthe pixel of interest, a pixel having a color component different fromthat of the pixel of interest. Thus, the camera can output an imagesignal that is equivalent to one output by the CCD of a three-platecamera.

FIG. 8 shows the structure of a prediction tap. The prediction tap iscomposed of 9 pixels forming a 3.times.3 matrix, the central pixel beingthe pixel of interest (i.e., the pixel to be processed). Theblock-generating section 28 extracts the prediction tap for each pixelof interest, thereby generating a plurality of blocks, each consistingof 9 pixels, the central one of which is the pixel of interest thatcorresponds to the prediction tap. The pixel of interest may be any oneof the pixels that constitute one frame.

The adaptation process section 31 performs the adaptation process. Ifthe pixel signal representing the pixel of interest is an R signal, an Rsignal, a G signal and a B signal will be generated at the position ofthe/pixel represented by the R signal. If the pixel signal is a G signalor a B signal, an R signal, a G signal and a B signal will be generatedat the position of the pixel represented by the G signal or the Bsignal. Consider a one-frame image signal that is composed of 8.times.6pixels as shown in FIG. 9A. When the 8.times.6 pixels sequentiallysubjected to the adaptation process, each as a pixel of interest,8.times.6 R signals are obtained as shown in FIG. 9B, 8.times.6 Gsignals are obtained as shown in FIG. 9C, and 8.times.6 B signals areobtained as shown in FIG. 9D. In other words, an image signal isgenerated, which is equivalent to one output by the CCD of a three-platecamera.

In the digital still camera 1 according to this invention, aclassification-adaptation process is effected, generating color signalsR, G and B that are equivalent to the outputs of the CCD of athree-plate camera. Thus, the edge parts and fine parts of the imageincrease in sharpness, and the S/N ratio of the image signal increasesTo generate an R signal for the central pixel (G22) in the 3.times.3pixel matrix (G.sub.11, B.sub.12, B.sub.32 and G.sub.33) of FIG. 1A, theset of prediction coefficients (w.sub.1 to w.sub.9) shown in FIG. 10B isused. Note that the prediction coefficients (w.sub.1 to w.sub.9)correspond to 3.times.3 pixels (G.sub.11, B.sub.12, . . . B.sub.32,G.sub.33). In the set of prediction coefficients, shown in FIGS. 10A and10B, the pixel (G.sub.22) of interest is a G signal. Pixels (B.sub.12and B.sub.32), both being B signals, are arranged above and below thepixel (G.sub.22) of interest, i.e., a G signal, respectively. Pixels(R.sub.21 and R.sub.23), both being R signals, are arranged to the leftand right of the pixel (G.sub.22) of interest, respectively. Pixels(G.sub.11, G.sub.13, G.sub.31 and G.sub.33), each being G signal, arearranged at upper-left, upper-right, lower-left and lower-rightpositions with respect to the pixel (G.sub.22) of interest,respectively. The prediction coefficients of this set are used togenerate an R signal at the position of the G signal that is the pixel(G.sub.22) of interest.

To generate an R signal for the central pixel (R.sub.22) in the3.times.3 pixel matrix (G.sub.11, R.sub.12, R.sub.32 and G.sub.33) ofFIG. 11A, the set of prediction coefficients (w.sub.1 to w.sub.9) shownin FIG. 1B is used. Note that the prediction coefficients (w.sub.1 tow.sub.9) correspond to 3.times.3 pixels (G.sub.11, B.sub.12, . . .B.sub.32, G.sub.33). In the set of prediction coefficients, shown inFIGS. 11A and 11B, the pixel (G.sub.22) of interest is a G signal. An Rsignal will be generated at the position of this pixel (G.sub.22) ofinterest. Pixels (R.sub.12 and R.sub.32), both being R signals, arearranged above and below the pixel (G.sub.22) of interest, i.e., a Gsignal, respectively. Pixels (B.sub.21 and B.sub.23), both being Gsignals, are arranged to the left and right of the pixel (G.sub.22) ofinterest, respectively. The prediction coefficients (w.sub.1 to w.sub.9)of this set are used to generate an R signal at the position of the Gsignal that is the pixel (G.sub.22) of interest.

To generate an R signal for the central pixel (B.sub.22) in the3.times.3 pixel matrix (R.sub.11, G.sub.12, . . . , G.sub.32 andR.sub.33) of FIG. 12A, the set of prediction coefficients (w.sub.1 tow.sub.9) shown in FIG. 12B is used. Note that the predictioncoefficients (w.sub.1 to w.sub.9) correspond to 3.times.3 pixels(R.sub.11, G.sub.12, . . . , G.sub.32, R.sub.33). In the set ofprediction coefficients, shown in FIGS. 12A and 12B, the pixel(B.sub.22) of interest is a B signal. Pixels (G.sub.12 and G.sub.32),both being G signals, are arranged above and below the pixel (B.sub.22)of interest, i.e., a B signal, respectively. Pixels (G.sub.21 andG.sub.23), both being G signals, are arranged to the left and right ofthe pixel (B.sub.22) of interest, respectively. Pixels (R.sub.11,R.sub.13, R.sub.31 and R.sub.33), each being R signal, are arranged atupper-left, upper-right, lower-left and lower-right positions withrespect to the pixel (B.sub.22) of interest, respectively. Theprediction coefficients (w.sub.1 to w.sub.9) of this set are used togenerate an R signal at the position of the B signal that is the pixel(B.sub.22) of interest.

The sets of prediction coefficients, described above, have been acquiredby learning and are stored in the coefficient memory 32.

How the sets of prediction coefficients are acquired by learning will beexplained. FIG. 13 is a block diagram showing shows a learning apparatus40 that acquires sets of prediction coefficients by learning.

In the learning apparatus 40, an image signal is supplied, as a teacherimage signal, to an extraction section 41 and a teacher-image blockgenerating section 45. The teacher image signal has the same signalformat as the output signal to be generated as the result of aclassification-adaptation process. In other words, the teacher imagesignal is identical in format to an image signal that is equivalent tothe outputs of the CCD of a three-plate camera. The extraction section41 extracts pixels from the teacher-image signal, in accordance with thearrangement of the color filters constituting an array. Pixels areextracted from the teacher-image signal by using filters equivalent tothe optical low-pass filters, with respect to the CCD image sensor 5.That is, pixels are extracted in consideration of the optical systemactually employed. The output of the extraction section 41 is astudent-image signal, which is supplied to a student-image blockgenerating section 42.

The student-image block generating section 42 extracts the class tap andprediction tap related to the pixel of interest, from the student-imagesignal generated by the extraction section 41, while referring to therelation between the teacher-image signal and the predicted pixel foreach block. The section 42 converts the student-image signal to a block,which is supplied to an ADRC process section 43 and an operating section46. The ADRC process section 43 performs an ADRC process on thestudent-image signal supplied from the student-image block generatingsection 42, thereby generating characteristic data. The characteristicdata is supplied to a classification process section 44. Theclassification process section 44 generates a class code from thecharacteristic data input to it. The class code is output to anoperating section 46.

The teacher-image signal is an image signal that has a resolutionsimilar to that of the CCD output of a single-plate camera. In otherwords, the teacher-image signal has a lower resolution than a imagesignal generated by a three-plate camera. Thus, the teacher-image signalrepresents pixels, each consisting of a R component, a G component and aB component. By contrast, the student-image signal represents pixels,each consisting of only one of the R, G and B components,

The teacher-image block generating section 45 extracts the image signalrepresenting the predicted pixel, from the teacher-image signal, whilereferring to the class tap of the student-image signal. The relationbetween the class tap teacher-image signal and the predicted pixel foreach block. The image signal extracted and representing the predictedpixel is supplied to the operating section 46. The operating section 46carries out an operation on the class number supplied from theclassification process section 44, while maintaining the relationbetween the image signal of the prediction tap supplied from thestudent-image block generating section 42 and the predicted imagesupplied from the teacher-image block generating section 45. Thus, theoperating section 46 generates the data of a normal equation, thesolution of which is a set of prediction coefficients. The data of thenormal equation, generated by the operating section 46, is sequentiallyread into a learned data memory 47 and held therein.

Another operating section 48 is provided, which solves a normal equationby using the data stored in the learned data memory 47. The set ofprediction coefficients is thereby calculated for one class. The set ofprediction coefficients, thus calculated, is stored into a coefficientmemory 49 and associated with the class. The contents of the coefficientmemory 49 are loaded into the coefficient memory 32 described above, andwill be utilized to achieve the classification-adaptation process.

How the learning apparatus 40 functions will be explained with referenceto the flowchart of FIG. 14.

The digital image signal input to the learning apparatus 40 representsan image that is comparable, in terms of quality, with an image pickedup by a three-plate camera. An image signal generated by a three-platecamera (i.e., a teacher-image signal) represents pixels, each havingthree primary-color signals, R, G and B. On the other hand, an imagesignal generated by a single-plate camera (i.e., a student-image signal)represents pixels, each having only one of three primary-color signals,R, G and B. The teacher-image signal input to the learning apparatus 40is, for example, a teacher-image signal that has been generated byfiltering an HD image signal output by a three-plate camera andillustrated in FIG. 15A and then converting the HD signal to a 1/4-sizeHD image signal shown in FIG. 15B.

In Step S31, the teacher-image block generating section 45 converts theinput teacher-image signal to a block. In Step S31, too, thestudent-image block generating section 42 extracts the value of thepredicted pixel located at a position corresponding to the pixel to bedesignated as the pixel of interest and supplies the pixel value to theoperating section 46.

In Step S32, the extraction section 41 thins the teacher-image signalthat represents an image comparable, in quality, with an image picked upby a three-plate camera, by effecting a filtering process equivalent toone accomplished by the color-coding filter 5 provided in the CCD imagesensor 5 of the single-plate camera. The section 41 generates astudent-image signal of the type shown in FIG. 15C, which corresponds tothe image signal output from the CCD image sensor 5 incorporated in thesingle-plate camera. The student-image signal, thus generated, is outputto the student-image block generating section 42.

In Step S33, the student-image block generating section 42 converts theinput student-image signal to a block. The section 42 generates a classtap and a prediction tap for the block, on the basis of the pixel ofinterest.

In Step S34, the ADRC process section 43 carries out the ADRC process onthe color signals of the class extracted from the student-image signal.

In Step S35, the classification process section 44 classifies theresults of the ADRC process and generates a signal representing theclass number assigned to the ADRC results classified.

In Step S36, the operating section 46 generates the above-mentionednormal equation (10) for the class number supplied from theclassification process section 44, on the basis of the prediction tapsupplied from the student-image block generating section 42 and thepredicted image supplied from the teacher-image block generating section45. The normal equation (10) is stored into the learned data memory 47.

In Step S37, it is determined whether the operating section 46 hasfinished processing all blocks or not. If there are any blocks notprocessed yet, the operation returns to Step S36. In this case, StepsS36 and S37 are repeated. If it is determined that all blocks have beenprocessed, the operation goes to Step S38.

In Step S38, the operating section 48 solves the normal equation storedin the learned data memory 47 by means of, for example, the Gauss-Jordanelimination or the Kolensky decomposition, thereby calculating a set ofprediction coefficients. The set of prediction coefficients, thuscalculated, is stored into the coefficient memory 49 and associated withthe class code output from the classification process section 44.

In Step S39, it is determined whether or not the operating section 48has solved the normal equation for all classes. If the equation has notbeen solved for any classes, the operation returns to Step S38. In thiscase, Steps S38 and S39 are repeated.

If it is determined in Step S39 that the normal equation has been solvedfor all classes, the operation is terminated.

The set of prediction coefficients, which is stored in the coefficientmemory 49, associated with the class code, is stored into thecoefficient memory 32 of the image-processing section 8 illustrated inFIG. 4. The adaptation process section 31 provided in the image-signalprocessing section 8 utilizes the set of prediction coefficients, heldin the coefficient memory 32, carrying out an adaptation process on thepixel of interest, by using the linear combination model expressed bythe equation (3).

FIGS. 16A to 16C, FIGS. 17A to 17C, FIGS. 18A to 18C and FIGS. 19A to19C show class taps. The class taps are used to determine classes whenan image signal color-coded by the color-filter array of Bayerarrangement is processed to generate an R signal, a G signal or a Bsignal at the position of the pixel of interest (the pixel shaded in thefigure). The class taps are also used when a set of predictioncoefficients is calculated to be applied in the process of generatingthe R, G or B signal.

The class taps 1 shown in FIGS. 16A to 16C include two R signals each.The R signals are arranged to the left and right of a G signal that isthe pixel of interest.

The class tap 1 shown in FIG. 16A is used to calculate a set ofprediction coefficients that will be applied to generate an R signal atthe position of the G signal for the pixel of interest. The class tap 1is composed of eight pixels. More precisely, it consists of two R-signalpixels arranged to the left and right of the G-signal pixel (i.e., thepixel of interest), two R-signal pixels arranged above and below thefirst R-signal pixel and spaced apart therefrom by one pixel-distance,respectively, two R-signal pixels arranged above and below the secondR-signal pixel and spaced apart therefrom by a one-pixel distance,respectively, and two R-signal pixel arranged to the left of the firstR-signal pixel and right of the second R-signal pixel and spaced aparttherefrom by a one-pixel distance, respectively.

The class tap 1 shown in FIG. 16B is used to calculate a set ofprediction coefficients that will be applied to generate a G signal atthe position of the G signal for the pixel of interest This class tap 1is composed of nine pixels. More precisely, it consists of the G-signalpixel of interest, four G-signal pixels arranged at upper-left,upper-right, lower-left and lower-right positions with respect to theG-signal pixel of interest, respectively, two G-signal pixels arrangedabove and below the G-signal pixel of interest and spaced aparttherefrom by one pixel-distance, respectively, and two G-signal pixelsarranged to left and right of the G-signal pixel of interest and spacedapart therefrom by a one-pixel distance, respectively.

The class tap 1 shown in FIG. 16C is used to calculate a set ofprediction coefficients that will be applied to generate a B signal atthe position of the G signal for the pixel of interest. The class tap 1is composed of eight pixels. More specifically, it consists of twoB-signal pixels arranged above and below the G-signal pixel of interest,respectively, two B-signal pixels arranged to left and right of thefirst B-signal pixel and spaced by a one-pixel distance, two B-signalpixels arranged to left and right of the second B-signal pixel andspaced by a one-pixel distance, two B-signal pixels arranged above thefirst B-signal pixel and below the second B-signal pixel and spacedtherefrom by a one-pixel distance, respectively.

The class taps 2 shown in FIGS. 17A to 17C include two R signals each.The R signals are arranged to the left and right of a G signal that isthe pixel of interest.

The class tap 2 shown in FIG. 17A is used to calculate a set ofprediction coefficients that will be applied to generate an R signal atthe position of the G signal for the pixel of interest. The class tap 2is composed of eight pixels. To be more specific, it consists of twoR-signal pixels arranged above and below the G-signal pixel of interest,respectively, two R-signal pixels arranged above the first R-signalpixel and below the second R-signal pixel and spaced therefrom by aone-pixel distance, respectively, two R-signal pixels arranged to theleft and right of the first R-signal pixel and spaced therefrom by aone-pixel distance, respectively, and two R-signal pixels arranged tothe left and right of the second R-signal pixel and spaced therefrom bya one-pixel distance, respectively.

The class tap 2 shown in FIG. 17B is used to calculate a set ofprediction coefficients that will be applied to generate a B signal atthe position of the G signal for the pixel of interest. The class tap 2is composed of nine pixels. More correctly, it consists of the G-signalpixel of interest, four G-signal pixels arranged at upper-left,upper-right, lower-left and lower-right positions with respect to theG-signal pixel of interest, respectively, two G-signal pixels arrangedabove and below the G-signal pixel it consists of two B-signal pixelsarranged to the left and right of the G-signal pixel of interest, twoB-signal pixels arranged above and below the first B-signal pixel andspaced apart therefrom by one pixel-distance, two B-signal pixelsarranged above and below the second B-signal pixel and spaced aparttherefrom by a one-pixel distance, and two B-signal pixels arranged tothe left of the first B-signal pixel and left of the second B-signalpixel and spaced apart therefrom by a one-pixel distance, respectively.

The class taps 3 shown in FIGS. 18A to 18C include two R signals each.The G signals are arranged to the left and right of a B signal that isthe pixel of interest.

The class tap 3 shown in FIG. 18A is used to calculate a set ofprediction coefficients that will be applied to generate an R signal atthe position of the G signal for the pixel of interest. The class tap 3is composed of eight pixels. More precisely, it consists of fourR-signal pixels arranged at upper-left, upper-right, lower-left andlower-right positions with respect to the G-signal pixel of interest,respectively, an R-signal pixel arranged above the first R-signal pixeland spaced therefrom by a one-pixel distance, an R-signal pixel arrangedto the left of the second R-signal pixel and spaced therefrom by aone-pixel distance, an R-signal pixel arranged to the below the thirdR-signal pixel and spaced therefrom by a one-pixel distance, and anR-signal pixel arranged to the right of the fourth R-signal pixel andspaced therefrom by a one-pixel distance.

The class tap 3 shown in FIG. 18B is used to calculate a set ofprediction coefficients that will be applied to generate a G signal atthe position of the B signal for the pixel of interest. This class tap 3is composed of eight pixels. To be more specific, it consists of fourR-signal pixels arranged at upper-left, upper-right, lower-left andlower-right positions with respect to the G-signal pixel of interest,respectively, a G-signal pixel arranged at an upper-left position withrespect to the first G-signal pixel, a G-signal pixel arrange at anlower-left position with respect to the second G-signal pixel, aG-signal pixel arranged at an lower-right position with respect to thethird G-signal pixel, and a G-signal pixel arranged at an upper-rightposition with respect to the fourth G-signal pixel.

The class tap 3 shown in FIG. 18C is used to calculate a set ofprediction coefficients that will be applied to generate a B signal atthe position of the B signal for the pixel of interest. This class tap 3is composed of nine pixels. To be more specific, it consists of theB-signal pixel of interest, two B-signal pixels arranged above and belowthe B-signal pixel of interest and spaced therefrom by a one-pixeldistance, respectively, two B-signal pixels arranged to left and writeof the B-signal pixel of interest and spaced therefrom by a one-pixeldistance, respectively, and four B-signal pixels arranged at upper-left,upper-right, lower-left and lower-right positions with respect to theG-signal pixel of interest and spaced therefrom by a one-pixel distance,respectively.

The class taps 4 shown in FIGS. 19A to 19C include two G signals each.The G signals are arranged to the left and right of an R signal that isthe pixel of interest.

The class tap 4 shown in FIG. 19A is used to calculate a set ofprediction pixels arranged at upper-left, upper-right, lower-left andlower-right positions with respect to the G-signal pixel of interest andspaced therefrom by a one-pixel distance, respectively.

The class taps 4 shown in FIGS. 19A to 19C include two G signals each.The G signals are arranged to the left and right of an R signal that isthe pixel of interest.

The class tap 4 shown in FIG. 19A is used to calculate a set ofprediction coefficients that will be applied to generate an R signal atthe position of the R signal for the pixel of interest. The class tap 4is composed of nine pixels. More correctly, it consists of the R-signalpixel of interest, two R-signal pixels arranged a above and below theR-signal pixel of interest and spaced apart therefrom by a one-pixeldistance, respectively, two R-signal pixels arranged to the left andright of the R-signal pixel of interest and spaced apart therefrom by aone-pixel distance, respectively, and four R-signal pixels arranged atupper-left, upper-right, lower-left and lower-right positions withrespect to the R-signal pixel of interest and spaced therefrom by aone-pixel distance, respectively.

The class tap 4 shown in FIG. 19B is applied to calculate a set ofprediction coefficients that will be applied to generate a G signal atthe position of the R signal for the pixel of interest. This class tap 4is composed of eight pixels. More correctly, it has of four G-signalpixels arranged at upper-left, upper-right, lower-left and lower-rightpositions with respect to the R-signal pixel of interest, respectively.Further, it has a G-signal pixel arranged at an upper-left position withrespect to the first G-signal pixel and spaced therefrom by a one-pixeldistance, a G-signal pixel arrange at an lower-left position withrespect to the second G-signal pixel and spaced therefrom by a one-pixeldistance, a G-signal pixel arranged at an lower-right position withrespect to the third G-signal pixel and spaced therefrom by a one-pixeldistance, and a G-signal pixel arranged at an upper-right position withrespect to the fourth G-signal pixel and spaced therefrom by a one-pixeldistance.

The class tap 4 shown in FIG. 19C is used to calculate a set ofprediction coefficients that will be applied to generate a B signal atthe position of the R signal for the pixel of interest. The class tap 4is composed of eight pixels. More specifically, it has four B-signalpixels arranged at upper-left, upper-right, lower-left and lower-rightpositions with respect to the R-signal pixel of interest, respectively.The class tap 4 further has a B-signal pixel arranged above the firstB-signal pixel and spaced therefrom by a one-pixel distance, a B-signalpixel arrange to the left of the second B-signal pixel and spacedtherefrom by a one-pixel distance, a B-signal pixel arranged below thethird B-signal pixel and spaced therefrom by a one-pixel distance, and aB-signal pixel arranged to the right of the fourth B-signal pixel andspaced therefrom by a one-pixel distance.

In the digital still camera 1, the block-generating section 28 extractsa class tap from the pixels that have been extracted by using theabove-mentioned class taps 1 to 4, in accordance with the color of thepixel of interest and the color of the pixel generated at the positionof the pixel of interest. The ADRC process section 29 performs the ADRCprocess on the class tap thus extracted, thereby generatingcharacteristic data. Then, the classification process section 30classifies the characteristic data, generating a class number (classcode). The set of prediction coefficients, which corresponds to theclass number, is read from the coefficient memory 32. The adaptationprocess section 31 utilizes the set of prediction coefficients thus readand effects an adaptation process on the pixel of interest, by using thelinear combination model expressed by the equation (3). A pixel having acolor component different from all color components can, therefore, begenerated at the position of the pixel of interest.

As described above, a class tap and a prediction tap are extracted onthe basis of the pixel of interest, which is contained in an input imagesignal. A class code is then generated from the class tap extracted.Further, the set of prediction coefficients corresponding to the classcode and the prediction tap extracted are used, generating a R signal, aG signal and a B signal at the position of the pixel of interest. Hence,an image signal of high resolution can be obtained.

Moreover, a student-image signal is generated from a teacher-imagesignal input, and a class tap is extracted on the basis of the pixel ofinterest, which is contained in the student-image signal. The pixelvalue of the teacher-image signal, which is located at the positioncorresponding to the pixel of interest in the student-image signal, isextracted. A class code is generated from the class tap extracted. Theclass tap and the pixel value, extracted, are used, thereby calculatinga set of prediction coefficients that will be applied to the operationfor generating a new color signal at the position of the pixel ofinterest, which is contained in the student-image signal. The set ofprediction coefficients and the class code are stored in a memory,associated with each other. The set of prediction coefficients, thuscalculated, can therefore be used in an image-signal processingapparatus that processes image signals to provide images of highresolution.

As described above, the prediction tap the image-signal processingsection 8 uses to perform the adaptation process and the class tap thelearning apparatus 40 uses to calculate a set of prediction coefficientsare of different structures. The prediction tap and the class tap may,nonetheless, be of the same structure. Furthermore, the structures ofthe prediction tap and class tap are not limited to those describedabove.

As has been indicated, the color-coding filter 4 is one having acolor-filter array of Bayer arrangement. Any other type of acolor-coding filter in the present invention can of course replace thefilter 4.

FIG. 20A to FIG. 20N show various color-filter arrays that may be usedin the color-coding filter 4 incorporated in the CCD image sensor 5 ofthe digital still camera.

FIGS. 20A to 20G show four color-filer arrays that may be used in thecolor-coding filter 4, each having green (G) filters, red (R) filtersand blue (B) filters.

FIG. 20A shows a Bayer arrangement of color filters. FIG. 20B depicts anin-line arrangement. FIG. 20C shows a G-stripe, RB chessboardarrangement. FIG. 20D shows a G-stripe, RB perfect chessboardarrangement. FIG. 20E illustrates a stripe arrangement. FIG. 20F depictsa slant stripe arrangement. FIG. 20G illustrates a primarycolor-difference arrangement.

FIGS. 20H to 20N illustrate various arrangements of color-filter arrayswhich may be used in the color-coding filter 4, each comprising magenta(M) color filters, yellow (Y) color filters, and cyan (C) color filersand white (W) color filters that pass complementary-color components oflight. FIG. 20H shows a field color-difference sequence arrangement.FIG. 201 depicts a frame color-difference sequence arrangement. FIG. 20Jshows a MOS-type arrangement. FIG. 20K illustrates a modified MOS-typearrangement. FIG. 20L shows a frame interleave arrangement. FIG. 20Mdepicts a field interleave arrangement. FIG. 20N shows a striparrangement.

The complementary color components (M, Y, C, W and G) are defined asfollows:Y=G+RM=R+BC=G+BW=R+G+B

The color components (YM, YG, CM and CG) of light that pass through thecolor-coding filter 4 of the frame color-difference sequencearrangement, shown in FIG. 201, are given as:YM=Y+M−2R+G+BCG=C+G=2G+BYG=Y+G=R+2GCM=C+M=R+G+2R

The image signal generated by performing the adaptation process on animage signal output from the CCD of a single-plate camera and equivalentto an image signal output from the CCD of a three-plate camerarepresents an image more sharp than an image represented by an imagesignal generated by the conventional linear interpolation. Hence, if theimage signal is further subjected to interpolation to provide an imagein density four times as high, the resultant image will have asufficient sharpness. For example, an image signal showing n.times.m Rpixels, an image signal representing n.times.m G pixels and an imagesignal showing n.times.m B pixels, which are shown in FIG. 21B, aregenerated by the adaptation process from the image signal output fromthe CCD image sensor that is composed of n.times.m pixels (n and m arepositive integers) as illustrated in FIG. 21A. That is, an image signalequivalent to one output from the CCD of a three-plate camera isgenerated by means of adaptation process, an image signal for2n.times.2m R pixel is generated from the signal for n.times.m R pixelsby means of prediction process, an image signal for 2n.times.2m B pixelsis generated from the signal for n.times.m G pixels by means ofprediction process, and an image signal for 2n.times.2m B pixels isgenerated from the signal for n.times.m B pixels by prediction process.Thus, a sharp image can be obtained in density four times as high.

The image-signal processing section 8 incorporated in the digital stillcamera 1 can generate an image signal of a higher density (fourfolddensity in this instance), directly from the output of the CCD imagesensor 5 by means of the adaptation process.

The image-signal processing section 8 performs the adaptation processon, for example, an image signal output from the CCD image sensor andcomposed of n.times.m pixels (n and m are positive integers) shown inFIG. 22A. Thus, the section 8 generates an image signal of n.times.m Gpixels, an image signal of n.times.m B pixels, and an image signal ofn×m R pixels, as is illustrated in FIG. 22B, directly from the imagesignal.

In the image-signal processing section 8, the block-generating section28 divides the input image signal into p.times.q blocks (p and q arepositive integers). The ADRC process section 29 extracts a class tapfrom each block, as will be described below, and performs the ADRCprocess on the class tap.

FIGS. 23 to 26 show examples of class taps. These class taps are of thetypes applied in the case where the color-filter array of the CCD imagesensor 5 has the Bayer arrangement. FIG. 23 shows a class tap used togenerate R, G and B pixels of fourfold density, around one R pixel. Inthe figure, marks.times.are added to the pixel that is to be generated,and the class tap is framed with thick lines.

FIGS. 24 and 25 show class taps, each being of the type used to generateR, G and B pixels of fourfold density, around one G pixel. Moreprecisely, the class tap shown in FIG. 24 is applied to generate pixelsaround a G pixel present in a row in which R pixels exist. The class tapshown in FIG. 25 is used to generate R, G and B pixels of fourfolddensity, around a G pixel present in a row in which B pixels exist.

FIG. 26 shows a class tap which is used to generate R, G and B pixels offourfold density, around a B pixel present in a row in which B pixelsexist.

In order to generate an image signal representing four pixelssurrounding an R pixel, each having R, G or B component, the ADRCprocess section 29 extracts a class tap that consists of those of 26pixels framed in FIG. 23 which are of the corresponding colors. Thesection 29 carries out the ADRC process on the signal values of the R, Gand G components of each pixel of the class tap.

In Step S17, the classification process section 30 classifies thesignals supplied from the ADRC process section 31. That is, the section30 determines a class for the those of 26 pixels framed in FIG. 23 whichhave been extracted and which correspond to the signal values obtainedby the ADRC process. The number of the class thus determined is suppliedto the adaptation process section 31. In Step S18, the adaptationprocess section 31 reads from the coefficient memory 32 the set ofcoefficients which corresponds to the class number supplied from theclassification process section 30. The section 31 multiplies thecoefficients of the set by the corresponding prediction tap. Theresultant products are added together, thereby generating afourfold-density image signal.

FIGS. 27 to 30 various prediction taps. FIG. 27 shows a prediction tapused to generate four pixels of fourfold density around one R pixel.FIGS. 28 and 29 show prediction taps, each applied to generate fourpixels of fourfold density, around one G pixel. To be more specific,FIG. 28 shows a prediction tap used to generate pixels of fourfolddensity, around a G pixel present in a row in which R pixels exist. FIG.29 shows a prediction tap used to generate fourfold-density pixelsaround a G pixel present in a row in which B pixels exist.

FIG. 30 depicts a prediction tap that is used to generate four pixels offourfold density, around one B pixel.

As clearly seen from FIGS. 27 to 30, any prediction tap used in thepresent embodiment is composed of 5.times.5 pixels located around thepixel of interest that corresponds to four pixels of fourfold density.

To predict the fourfold-density pixel, which is located at theupper-left position with respect to one R pixel as shown in FIG. 27, theadaptation process section 31 multiplies the prediction coefficients ofthe set identified by the class number supplied from the classificationprocess section 30, by the prediction tap. The resultant products areadded together. Such multiplication and addition are carried out, thuspredicting three pixels which are located at the upper-right, lower-leftand lower-right positions with respect to the R pixel.

The classification-adaptation process described above generates an imagesignal composed of 2n.times.2m R pixels, an image signal composed of2n.times.2m G pixels and an image signal composed of 2n.times.2m Bsignals, directly from a one-frame image signal consisting of n.times.mpixels (each being a signal representing one color only). This makes itpossible to provide an image more sharp than the image represented by animage signal of fourfold density, from an image signal composed ofn.times.m R pixels, an image signal composed of n.times.m G pixels andan image signal composed of n.times.m B pixels.

In the learning apparatus 40 converts the fourfold-density image signalto a teacher-image signal. The extraction section 41 performs anextraction process on the input teacher-image signal, so that colorfilters may be used which have a magnification power inverse to themagnification power the photographing system should have. Astudent-image signal is thereby generated which corresponds to the imagesignal output by the image signal output by the CCD image sensor 5 ofthis single-plate camera. Thus, it is possible to obtain the set ofprediction coefficients, described above.

To state it more specifically, in the learning apparatus 40, astudent-image signal is generated, in which a prescribed number ofsample values representing color components at pixel positions,respectively. A plurality of pixels located near the pixel of interestincluded in the predicted image signal that has a greater sample valuethan the student-image signal are extracted from the student-imagesignal thus extracted and representing a one-frame image. The predictedimage signal is classified on the basis of the pixels extracted.Further, a plurality of pixels located near the position of the pixel ofinterest are extracted from the teacher-image signal which correspondsto the predicted image signal and having color components at the pixelpositions. A set of prediction coefficients is thereby generated, foreach class, which corresponds to the above-mentioned student-imagesignal. The set of prediction coefficients, thus generated, will be usedin the prediction process to obtain an image signal corresponding to theteacher-image signal.

In order to evaluate the operating efficiency of the embodimentdescribed above, simulation was conducted on nine high-vision images ofthe ITE (Institute of Television Engineers) standard, using acolor-filter array of the Bayer arrangement. Further, the ninehigh-vision images were used, also to calculate a set of predictioncoefficients.

An image signal equivalent to an output of the CCD of a three-platecamera was subjected to an extraction operation in which themagnification of the classification-adaptation process was applied andthe positional relation of pixels was taken into account. An imagesignal equivalent to an output of the CCD of a single-plate camera wasthereby generated. The output of the CCD of the single-plate camera wasconverted to an image signal having twice as many pixels in both the lowdirection and the column direction, by means of aclassification-adaptation process in which the set of predictioncoefficients for the CCD output was utilized. The class tap andprediction tap, used in the classification-adaptation process, werethose illustrated in FIG. 23 to FIG. 30. The R, G and B pixels of theclass tap are processed independently of one another, whereas the R, Gand B pixels of the prediction tap are mixed and used.

The simulation resulted in an image signal that was sharp at edges andfine parts and exhibited a high resolution. The image signal was moresharp and had a higher resolution than a fourfold-density image signalgenerated from an output of the CCD of a single-plate camera (FIG. 21A)or from an output of the CCD of a single-plate camera (FIG. 21B).Simulation was conducted by means of linear interpolation, too. Theimage signal generated by classification process was found superior tothe image signal provided by linear interpolation in terms of resolutionand S/N ratio.

In the embodiment described above, the output of the single-plate CCDimage sensor is converted to a fourfold-density image signal.Nevertheless, the present invention can be applied to generate an imagesignal of any other density.

Moreover, the present invention can be applied not only to a digitalstill camera, but also to a video camera and any other type of animage-processing apparatus.

Various types of color-filter arrays are available, which can be used asthe color-coding filter 4 provided in the CCD image sensor 5 of thedigital still camera 1 described above. One signal value may representmore or less information than another signal value in the color-filterarray. In this case, the precision of the prediction process will varyif the color signals are subjected, one by one, to theclassification-adaptation process. If the color-filter array used is,for example, of the Bayer arrangement and if theclassification-adaptation process is performed on the R, G and Bsignals, independently of one another, the R signal and the B signalwill be processed in the same way as the G signal (two existing in everyfour pixels), though one R signal and one B signal exist in every fourpixels. As a consequence, the precision of the prediction process islower for the R and B signals than for the G signal.

Thus, if one signal value may represent more or less information thananother signal value, the color component arranged in a higher densitythan any other color component is applied to extract a plurality ofpixels for the pixel of interest of the input image signal, which hasany one of the color components. The pixels extracted are those whichhave a color component of higher density than any other color componentand which are located near the pixel of interest. The class of the inputimage signal is determined from the pixels thus extracted. On the basisof the class determined, a pixel having a color component different fromthe color component the pixel of interest has. The precision of theprediction process can thereby be enhanced.

The class tap and the prediction tap, which are applied to the imagesignal subjected to color coding in the color-filter array of the Bayerarrangement, will be described in detail. As shown in FIG. 31, forexample, a B pixel may be the pixel to be predicted. Then, as shown inFIG. 32, the class tap is composed of eight G pixels. Of these eight Gpixels, four are arranged above, below, to the left of and to the rightof, the pixel to be predicted, respectively. Two are arranged atupper-left and lower-left positions with respect to the G pixel on theleft of the pixel to be predicted, and the remaining two are arranged atupper-right and lower-right positions with respect to the G pixel on theright of the pixel to be predicted. In this case, the prediction tap iscomposed of 5.times.5 pixels, including R pixels, G pixels and B pixels,as illustrated in FIG. 33, with a B pixel located at the center of the5.times.5 matrix.

An R pixel may be the pixel to be predicted as is illustrated in FIG.34. In this case, the class tap is composed of eight G pixels as shownin FIG. 35. Of these eight G pixels, four are arranged above, below, tothe left of and to the right of, the R pixel to be predicted,respectively. Two are arranged at upper-left and lower-left positionswith respect to the G pixel on the left of the pixel to be predicted,and the remaining two are arranged at upper-right and lower-rightpositions with respect to the G pixel on the right of the pixel to bepredicted. The prediction tap in this case is composed of 25 pixels,including R pixels, G pixels and B pixels, as illustrated in FIG. 35,with an R pixel located at the center of the 5.times.5 matrix.

Further, a G pixel may be the pixel to be predicted as is illustrated inFIG. 37. In this case, the class tap is composed of nine G pixels asshown in FIG. 38A or FIG. 38B, in accordance with the colors of thepixels located near the pixel to be predicted. More precisely, the classtap shown in FIG. 38A is composed of the G pixel to be predicted, four Gpixels arranged above, below, to the left of and to the right of, the Gpixel to be predicted, respectively, two G pixels arranged above andblow the G pixel to be predicted and spaced therefrom by a R pixel, andtwo G pixels arranged to left and right of the G pixel to be predictedand spaced therefrom by a B pixel. The class tap shown in FIG. 38B iscomposed of the G pixel to be predicted, four G pixels arranged above,below, to the left of and to the right of, the G pixel to be predicted,respectively, two G pixels arranged above and blow the G pixel to bepredicted and spaced therefrom by a B pixel, and two G pixels arrangedto left and right of the G pixel to be predicted and spaced therefrom byan R pixel. The prediction tap in this case is composed of 5.times.5pixels, including R pixels, G pixels and B pixels, as illustrated inFIG. 39A or FIG. 39B, with a G pixel used as the pixel to be predicted.

A luminance signal Y is generated from the signal values for R, G and B,as in indicated by the following equation:Y=0.59G+0.30R+0.11B

As seen from this equation, the G component more influences theluminance signal Y than the other color components. Therefore, as shownin FIG. 31, G pixels are arranged more densely than the R pixels and theB pixels in the Bayer arrangement. The luminance signal Y contains agreat amount of information that influences the characteristic of humanvisual sense and the image resolution.

In view of this, it may be possible to effect theclassification-adaptation process with higher precision if the class tapis composed of only G pixels of the image signal.

To this end, the learning apparatus 40 extracts a plurality of pixelswhich are located near the pixel of interest of a student-image signalhaving at each pixel position, a color component representing any one ofthe colors and which have a color component more dense than any othercolor components. The class of the input image signal is determined fromthe pixels thus extracted. Thus, it is possible to obtain a set ofprediction coefficients that can be applied to accomplish the adaptationprocess described above.

To evaluate the operating efficiency of the embodiment described above,simulation was conducted on nine high-vision images of the ITE(Institute of Television Engineers) standard, using a color-filter arrayof the Bayer arrangement. Further, the nine high-vision images were usedto calculate a set of prediction coefficients. An image signalequivalent to an output of the CCD of a three-plate camera was subjectedto an extraction operation in which the magnification of theclassification-adaptation process was applied and the positionalrelation of pixels was taken into account. An image signal equivalent toan output of the CCD of a single-plate camera was thereby generated. Aset of prediction coefficients was generated by applying algorithmsimilar to the one that is used in the learning apparatus 40. Further,the output of the CCD of the single-plate camera was converted to animage signal having twice as many pixels in both the low direction andthe column direction, by means of the classification-adaptationdescribed above. The simulation resulted in an image signal that wasmore sharp at edges and had a higher resolution than in the case wherethe class tap applied to predict an R pixel (alternatively, a G signalor a B signal) is a pixel that has R, G and B components mixed together.Simulation was conducted by means of linear interpolation, notclassification-adaptation process. The image signal generated byclassification process was found superior to the image signal providedby linear interpolation in terms of resolution and S/N ratio.

The R, G and B signals representing three standard images A, B and C,which were obtained by classification-adaptation process in which R, Gand B pixels were extracted independently as class taps, exhibited thefollowing S/N ratios:

Standard Image A

R:35.22 db

G:35.48 db

B:34.93 db

Standard Image B

R:32.45 db

G:32.40 db

B:29.29 db

Standard Image C

R:24.75 db

G:25.31 db

B:23.23 db

By contrast, the R, G and B signals representing three standard imagesA, B and C, which were obtained by classification-adaptation process inwhich a G pixel was used as a class tap, exhibited the following S/Nratios:

Standard Image A

R:35.38 db

G:35.48 db

B:35.13 db

Standard Image B

R:32.60 db

G:32.40 db

B:29.46 db

Standard Image C

R:24.99 db

G:25.31 db

B:23.79 db

As indicated above, only the pixel of the color component, which is moredensely arranged than any other pixels, is used as a class tap in thecase where the signal values represent different amounts of data. Theprecision of prediction can thereby be enhanced.

Moreover, the class of input image signal may be classified inaccordance with the results of the ADRC process performed on the classtap including a plurality of color signals extracted for each color ofthe color-filter array. In this case, the prediction process can beaccomplished with a higher precision. Image signals of high resolutioncan therefore be generated.

A plurality of pixels located near the pixel of interest are extractedfor each pixel position, from the input image signal generated by theCCD image sensor of the single-plate camera. An ADRC process is theneffected, thereby generating characteristic data. The characteristicdata is used as the space activity of the pixels of each colorcomponent, which have been extracted. The class of the image signal isdetermined from the characteristic data. In accordance with the classthus determined, the prediction-process section 25 performsclassification-adaptation process, thereby generating-a pixel having acolor component different from that of the pixel of interest, at theposition of the pixel of interest. An image signal equivalent to anoutput of the CCD of a three-plate camera can therefore be obtained.

That is, the classification process section 30 receives the output ofthe ADRC process section 29, i.e., an re-quantized code, and classifiesthe space activity, i.e., the level-distribution pattern of the imagesignal. The section 30 generates a class number that represents theresults of the classification. The space activity is classified moreappropriately, which helps to raise the precision of the predictionprocess.

The class tap and the prediction tap applied to the image signalcolor-coded by the color-filter array of Bayer arrangement will bedescribed in detail.

FIGS. 40A, 40B and 40C illustrate a tap that serves to generate an Rsignal, a G signal and a B signal at the position of a B pixel (an imagesignal having a B component, too, will be generated at the position ofthe B pixel).

FIGS. 41A, 41B and 41C show that serves to generate an R signal, a Gsignal and a B signal at the position of a G pixel (an image signalhaving a B component, too, will be generated at the position of the Bpixel). In FIGS. 40A to 40C and FIGS. 41A to 41C, the double circleindicates the pixel to be predicted. In other words, the double circleindicates the position of the pixel to be predicted. In FIGS. 40G and41B, the triangles represent the pixels to be extracted to constitute aclass tap, or the positions thereof In FIGS. 40G and 41B, the trianglesindicate the positions of the pixels to be extracted to form aprediction tap.

To generate an R signal, a G signal and a B signal at the position of aB pixel as is illustrated in FIG. 40A, the pixels specified in FIG. 40Bare extracted and used as a class tap. The class tap is composed of ninepixel (indicated by the triangles), i.e., the pixel to be predicted andthe eight pixels surrounding the pixel to be predicted. Since the classtap includes only one G pixel, the characteristic data to be used toachieve classification by means of the ADRC process cannot be extractedreliably. Nonetheless, more pixels than the pixels actually used as aclass tap are subjected to the ADRC process, making it possible toextract the characteristic data without fail.

The class tap is extracted as is shown in FIG. 40C. More specifically,the pixels at which R, G and B components will be mixed are extracted asa class tap. The set of prediction coefficients, provided for the class,are applied in weighting process, addition process and the like, therebypredicting an image signal at the position predicted.

In the case of the Bayer arrangement, the number of pixels for each Rsignal, the number of pixels for each G signal and the number of pixelsfor each B signal are 1, 2 and 1, respectively. Hence, a class tap andprediction tap of the same structures as those applied to predict andgenerate the R, G and B signals at the position of the R pixel can beutilized to predict and generate an R signal, a G signal and a B signalat the position of the R pixel.

To generate an R signal, a G signal and a B signal at the position of aG pixel as is illustrated in FIG. 41A, the pixels specified in FIG. 41Bare extracted and used as a class tap. The class tap is composed of ninepixel (indicated by the triangles), i.e., the pixel to be predicted andthe eight pixels surrounding the pixel to be predicted. For any colorsignal having a small number of pixels, the class tap is expanded toinclude those pixels that are indicated by squares in FIG. 41 B. An ADRCprocess is performed on the signals of the class tap. Only the centralpixel is extracted from the results of the ADRC process, therebyachieving classification. Furthermore, a process is effected to applythe relation between the R, G and B pixels to the results of theclassification. For example, the dynamic range of the ADRC processperformed on each signal, the results of threshold process, the maximumand minimum dynamic ranges of the ADRC process, and the like are addedto the image signal, in the form of data items consisting of severalbits. An image signal can thereby predicted with high precision andgenerated, which represents a high-resolution image.

A prediction tap is extracted as shown in FIG. 41C. More precisely, thepixels at which R, G and B components will be mixed are arranged as aclass tap. The set of prediction coefficients, provided for the class,are applied in weighting process, addition process and the like, therebypredicting an image signal at the position predicted.

To evaluate the operating efficiency of the embodiment described above,simulation was conducted on the assumption that a color-filter array ofthe Bayer arrangement is utilized. In the simulation, a set ofprediction coefficients was generated by applying algorithm similar tothe one that is used in the learning apparatus 40. Further, anextraction process was carried out, generating an image signalequivalent to an output of the CCD of a single-lens camera, from animage signal equivalent to an output of the CCD of a three-plate camera.Still further, a prediction process was implemented in theabove-mentioned classification-adaptation process. Moreover, simulationwas conducted by means of linear interpolation and by means of theclassification-adaptation process according to the invention. In theclassification-adaptation process, R, G and B pixels were classifiedindependently of one another. The results of the linear interpolationwere compared with the results of the classification-adaptation process.

More specifically, the simulation was conducted on nine high-visionimages of the ITE (Institute of Television Engineers) standard. The ninehigh-vision images were used, also to calculate a set of predictioncoefficients. The simulation resulted in an image signal that wassharper at edges and fine parts than the image signal generated by thelinear interpolation. In addition, it was confirmed that the S/N ratiohad improved. Moreover, the resolution of any images represented by R-and B-component image signals was higher than in the case where theclassification-adaptation process was effected on the R, G and B imagesignals independently. Thus, the embodiment of the present invention canprovide images that are superior, in terms of the sharpness of edges andfine parts, S/N ratio, resolution and the like, to those provided by thelinear interpolation or by classification-adaptation process wherein theR, G and B signals are classified independently.

Neither the class tap nor the prediction tap is limited to those shownin FIGS. 40A to 40C and FIGS. 41A to 41C. Rather, the class tap and theprediction tap may be changed in accordance with the arrangement of theprimary-color filter array or complementary-color filter array, thecharacteristics (e.g., resolution of the image signal to be generated,and the like. For example, more pixels may be extracted to constitute aclass tap or a prediction tap if it is demanded that an image of higherresolution be output.

As FIGS. 20A to 20N show, various types of color-filter arrays areavailable for the color-coding filter 4 incorporated in the CCD imagesensor 5 of the single-plate camera. The method described above iseffective and useful to color-filter array in which the informationitems represented by signal values differ in density.

The present invention can be applied, also to the case where an outputof the CCD image sensor is converted to an image signal having aresolution different from that of the CCD image sensor, for example byincreasing the pixels of each row and column of the CCD image sensor asshown in FIG. 22A or FIG. 22B, thus enhancing the resolution four times.That is, a learning process may be effected, using the image signal tobe generated as a teacher-image signal and the image signal output fromthe CCD image sensor 5 incorporated in the digital still camera 1,thereby generating a set of prediction coefficients. The predictioncoefficients, thus generated, may then be applied to theclassification-adaptation process.

The present invention can be applied to not only digital still cameras,but also movie cameras such as camera-incorporated VTRs, andimage-processing apparatus. Further, the invention can be applied toprinters, scanners, and the like.

The classification-adaptation process performed by theprediction-process section 25 and the learning process performed by thelearning apparatus 40 may be accomplished by an ordinary computer system310 shown in FIG. 43. As shown in FIG. 43, the computer system 310comprises a bus 311, a CPU (Central Processing Unit) 312, a memory 313,an input interface 314, a user interface 315, and an output interface316. The computer program that implements these processes is recorded ina recording medium. The computer program controls the computer, therebyprocessing input image signals, each representing a color component atone pixel. The recording medium storing this program is presented tousers, in the form of a magnetic disk, a CD-ROM or-the like. Moreover,the program can be transmitted to users through networks such as theInternet and digital-communications satellites.

1-27. (canceled)
 28. A learning apparatus comprising: first pixel-extracting means for extracting a plurality of pixels located near each pixel of interest of a student-image signal which has color components at positions of pixels, respectively; class-determining means for determining a class from the pixels extracted by the first pixel-extracting means; second pixel-extracting means for extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and prediction-coefficient generating means for generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted by the first pixel-extracting means and second pixel-extracting means.
 29. The learning apparatus according to claim 28, characterized in that the first pixel-extracting means extracts the pixels located near the pixel of interest, in accordance with the position of the pixel of interest and the color component of the pixel of interest.
 30. The learning apparatus according to claim 28, characterized in that the class-determining means performs an ADRC (Adaptive Dynamic Range Coding) process on the pixels extracted by the first pixel-extracting means, thereby to generate characteristic data and determines the class from the characteristic data.
 31. A learning method comprising: a first pixel-extracting step of extracting a plurality of pixels located near each pixel of interest of a student-image signal which has color components at positions of pixels, respectively; a class-determining step of determining a class from the pixels extracted in the first pixel-extracting step; a second pixel-extracting step of extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and a prediction-coefficient generating step of generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted in the first pixel-extracting step and second pixel-extracting step.
 32. The learning method according to claim 31, characterized in that in the first pixel-extracting step, the pixels located near the pixel of interest is extracted in accordance with the position of the pixel of interest and the color component of the pixel of interest.
 33. The learning method according to claim 31, characterized in that in the class-determining step, an ADRC (Adaptive Dynamic Range Coding) process is performed on the pixels extracted in the first pixel-extracting step, thereby to generate characteristic data and determines the class from the characteristic data.
 34. A recording medium storing a computer program designed to perform a learning process to generate a set of prediction coefficients corresponding to a class, said computer program comprising: a first pixel-extracting step of extracting a plurality of pixels located near each pixel of interest of a student-image signal which has color components at positions of pixels, respectively; a class-determining step of determining a class from the pixels extracted in the first pixel-extracting step; a second pixel-extracting step of extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and a prediction-coefficient generating step of generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted in the first pixel-extracting step and second pixel-extracting step.
 35. The recording medium according to claim 34, characterized in that in the first pixel-extracting step, the pixels located near the pixel of interest is extracted in accordance with the position of the pixel of interest and the color component of the pixel of interest.
 36. The learning method according to claim 34, characterized in that in the class-determining step, an ADRC (Adaptive Dynamic Range Coding) process is performed on the pixels extracted in the first pixel-extracting step, thereby to generate characteristic data and determines the class from the characteristic data. 37-42. (canceled)
 43. A learning apparatus comprising: first pixel-extracting means for extracting a plurality of pixels located near each pixel of interest from a student-image signal having a prescribed number of sample values which constitute one image and each of which represents any one of various colors at a position of a pixel, said pixel of interest being one included in an image to be predicted, which has more sample values than the prescribed number; class-determining means for determining a class from the pixels extracted by the first pixel-extracting means; second pixel-extracting means for extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest, from a teacher-image signal which corresponds to the image signal to be predicted and which have a plurality of color components for each pixel; and prediction-coefficient generating means for generating a set of prediction coefficients for each class, for use in a prediction process for generating an image signal corresponding to the teacher-image signal, from an image signal that corresponds to the student-image signal, in accordance with values of the pixels extracted by the first pixel-extracting means and second pixel-extracting means.
 44. The learning apparatus according to claim 43, characterized in that the class-determining means performs an ADRC (Adaptive Dynamic Range Coding) process on the pixels extracted by the first pixel-extracting means, thereby to generate characteristic data and determines the class from the characteristic data.
 45. A learning method comprising: a first pixel-extracting step of extracting a plurality of pixels located near each pixel of interest, from a student-image signal having a prescribed number of sample values which constitute one image and each of which represents any one of various colors at a position of a pixel, said pixel of interest being one included in an image to be predicted, which has more sample values than the prescribed number; a class-determining step of determining a class from the pixels extracted in the first pixel-extracting step; a second pixel-extracting step of extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest, from a teacher-image signal which corresponds to the image signal to be predicted and which have a plurality of color components for each pixel; and a prediction-coefficient generating step of generating a set of prediction coefficients for each class, for use in a prediction process for generating an image signal corresponding to the teacher-image signal, from an image signal that corresponds to the student-image signal, in accordance with values of the pixels extracted in the first pixel-extracting step and second pixel-extracting step.
 46. The learning method according to claim 45, characterized in that in the class-determining step, an ADRC (Adaptive Dynamic Range Coding) process is performed on the pixels extracted by the first pixel-extracting means, thereby to generate characteristic data and determines the class from the characteristic data.
 47. A recording medium storing a computer program designed to perform a learning process in accordance with a class, said computer program comprising: a first pixel-extracting step of extracting a plurality of pixels located near each pixel of interest, from a student-image signal having a prescribed number of sample values which constitute one image and each of which represents any one of various colors at a position of a pixel, said pixel of interest being one included in an image to be predicted, which has more sample values than the prescribed number; a class-determining step of determining a class from the pixels extracted in the first pixel-extracting step; a second pixel-extracting step of extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest, from a teacher-image signal which corresponds to the image signal to be predicted and which have a plurality of color components for each pixel; and a prediction-coefficient generating step of generating a set of prediction coefficients for each class, for use in a prediction process for generating an image signal corresponding to the teacher-image signal, from an image signal that corresponds to the student-image signal, in accordance with values of the pixels extracted in the first pixel-extracting step and second pixel-extracting step.
 48. The recording medium according to claim 47, characterized in that the class-determining means performs an ADRC (Adaptive Dynamic Range Coding) process on the pixels extracted by the first pixel-extracting means, thereby to generate characteristic data and determines the class from the characteristic data. 49-57. (canceled)
 58. A learning apparatus comprising: first pixel-extracting means for extracting a plurality of pixels located near each pixel of interest of a student-image signal which has any one of various color components at a position of each pixel, said pixels each having a color component of the highest density of all color components; class-determining means for determining a class from the pixels extracted by the first pixel-extracting means; second pixel-extracting means for extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and prediction-coefficient generating means for generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted by the first pixel-extracting means and second pixel-extracting means.
 59. The learning apparatus according to claim 58, characterized in that the class-determining means performs an ADRC (Adaptive Dynamic Range Coding) process on the pixels extracted by the first pixel-extracting means, thereby to generate characteristic data and determines the class from the characteristic data.
 60. A learning method comprising: a first pixel-extracting step of extracting a plurality of pixels located near each pixel of interest of a student-image signal which has any one of various color components at a position of each pixel, said pixels each having a color component of the highest density of all color components; a class-determining step of determining a class from the pixels extracted in the first pixel-extracting step; a second pixel-extracting step of extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and a prediction-coefficient generating step of generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted in the first pixel-extracting means and second pixel-extracting step.
 61. The learning method according to claim 60, characterized in that in the class-determining step, an ADRC (Adaptive Dynamic Range Coding) process is performed on the pixels extracted by the first pixel-extracting means, thereby to generate characteristic data and determines the class from the characteristic data.
 62. A recording medium storing a computer program designed to perform a learning process in accordance with a class, said computer program comprising: a first pixel-extracting step of extracting a plurality of pixels located near each pixel of interest of a student-image signal which has any one of various color components at a position of each pixel, said pixels each having a color component of the highest density of all color components; a class-determining step of determining a class from the pixels extracted in the first pixel-extracting step; a second pixel-extracting step of extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and a prediction-coefficient generating step of generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted in the first pixel-extracting means and second pixel-extracting step.
 63. The recording medium according to claim 62, characterized in that in the class-determining step, an ADRC (Adaptive Dynamic Range Coding) process is performed on the pixels extracted by the first pixel-extracting means, thereby to generate characteristic data and determines the class from the characteristic data. 64-75. (canceled)
 76. A learning apparatus comprising: first pixel-extracting means for extracting a plurality of pixels for each color component, from pixels located near each pixel of interest of a student-image signal having color components at positions of pixels, respectively; class-determining means for generating characteristic data about the pixels of each color component, from the pixels of each color component which have been extracted by the first pixel-extracting means and for determining a class from the characteristic data generated for each color component; second pixel-extracting means for extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and prediction-coefficient generating means for generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted by the first pixel-extracting means and second pixel-extracting means.
 77. The learning apparatus according to claim 76, characterized in that the class-determining means determines a class by using, as the characteristic data, a space activity of the pixels of each color component, which have been extracted by the first pixel-extracting means.
 78. The learning apparatus according to claim 77, characterized in that the class-determining means generates the space activity by performing ADRC (Adaptive Dynamic Range Coding) process on the pixels of each color component.
 79. A learning method comprising: a first pixel-extracting step of extracting a plurality of pixels for each color component, from pixels located near each pixel of interest of a student-image signal having color components at positions of pixels, respectively; a class-determining step of generating characteristic data about the pixels of each color component, from the pixels of each color component which have been extracted in the first pixel-extracting step and for determining a class from the characteristic data generated for each color component; a second pixel-extracting step of extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and a prediction-coefficient generating step of generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted in the first pixel-extracting step and second pixel-extracting step.
 80. The learning method according to claim 79, characterized in that in the class-determining step, a class is determined by using, as the characteristic data, a space activity of the pixels of each color component, which have been extracted in the first pixel-extracting step.
 81. The learning method according to claim 80, characterized in that in the class-determining step, a space activity is generated by performing ADRC (Adaptive Dynamic Range Coding) process on the pixels of each color component.
 82. A recording medium storing a computer program designed to perform a learning process in accordance with a class, said computer program comprising: a first pixel-extracting step of extracting a plurality of pixels for each color component, from pixels located near each pixel of interest of a student-image signal having color components at positions of pixels, respectively; a class-determining step of generating characteristic data about the pixels of each color component, from the pixels of each color component which have been extracted in the first pixel-extracting step and for determining a class from the characteristic data generated for each color component; a second pixel-extracting step of extracting a plurality of pixels located near positions corresponding to the position of the pixel of interest of the student-image signal, from a teacher-image signal which corresponds to the student-image signal and which have a plurality of color components for each pixel; and a prediction-coefficient generating step of generating a set of prediction coefficients for each class, for use in generating an image signal corresponding to the teacher-image signal from an image signal corresponding to the student-image signal, in accordance with values of the pixels extracted in the first pixel-extracting step and second pixel-extracting step.
 83. The recording medium according to claim 82, characterized in that in the class-determining step, a class is determined by using, as the characteristic data, a space activity of the pixels of each color component, which have been extracted in the first pixel-extracting step.
 84. The recording medium according to claim 83, characterized in that in the class-determining step, a space activity is generated by performing ADRC (Adaptive Dynamic Range Coding) process on the pixels of each color component. 